[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:33:45 dist-test-slave-gjw7 krb5kdc[8162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:33:45 dist-test-slave-gjw7 krb5kdc[8162](info): set up 2 sockets
Aug 12 00:33:45 dist-test-slave-gjw7 krb5kdc[8162](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 00:33:46.242758 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.005s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:33:46 dist-test-slave-gjw7 krb5kdc[8162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:33:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:33:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:33:46.449851 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41861
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:41861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:46.797631 8194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:46.797998 8194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:46.798096 8194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:46.805627 8194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:33:46.805757 8194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:46.805804 8194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:46.805861 8194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:33:46.805905 8194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:33:46.816444 8194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:41861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:46.817960 8194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:46.819401 8194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:46.832113 8202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:46.832176 8208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:46.832078 8203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:46.833087 8194 server_base.cc:1034] running on GCE node
I20260812 00:33:46.834031 8194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:46.835642 8194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:46.836858 8194 hybrid_clock.cc:648] HybridClock initialized: now 1786494826836824 us; error 101 us; skew 500 ppm
I20260812 00:33:46.837229 8194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:46 dist-test-slave-gjw7 krb5kdc[8162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494826, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:46.840925 8194 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:33:46.842614 8194 webserver.cc:460] Webserver started at http://127.7.244.126:41431/ using document root <none> and password file <none>
I20260812 00:33:46.843406 8194 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:46.843538 8194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:46.843858 8194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:46.847163 8194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "326b19b1751841758749a7575fe7a800"
format_stamp: "Formatted at 2026-08-12 00:33:46 on dist-test-slave-gjw7"
server_key: "c111f95497dcfacfce4f75811fb53510"
server_key_iv: "beaba54e1612be49c90ff080d74425f5"
server_key_version: "encryptionkey@0"
I20260812 00:33:46.847985 8194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "326b19b1751841758749a7575fe7a800"
format_stamp: "Formatted at 2026-08-12 00:33:46 on dist-test-slave-gjw7"
server_key: "c111f95497dcfacfce4f75811fb53510"
server_key_iv: "beaba54e1612be49c90ff080d74425f5"
server_key_version: "encryptionkey@0"
I20260812 00:33:46.853981 8194 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:33:46.858453 8226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:46.860199 8194 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 00:33:46.860436 8194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "326b19b1751841758749a7575fe7a800"
format_stamp: "Formatted at 2026-08-12 00:33:46 on dist-test-slave-gjw7"
server_key: "c111f95497dcfacfce4f75811fb53510"
server_key_iv: "beaba54e1612be49c90ff080d74425f5"
server_key_version: "encryptionkey@0"
I20260812 00:33:46.860672 8194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:46.887213 8194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:46.892591 8194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:46.893054 8194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:46.912674 8194 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:41861
I20260812 00:33:46.912686 8320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:41861 every 8 connection(s)
I20260812 00:33:46.914172 8194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:33:46.918437 8323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:46.920974 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 8194
I20260812 00:33:46.921283 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:33:46.923278 8145 external_mini_cluster.cc:1428] Setting key eb3bd37ebdf6d0e5e4655fab359f1f3a
I20260812 00:33:46.930830 8323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800: Bootstrap starting.
I20260812 00:33:46.934367 8323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:46.935520 8323 log.cc:824] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800: Log is configured to *not* fsync() on all Append() calls
I20260812 00:33:46.939142 8323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800: No bootstrap required, opened a new log
I20260812 00:33:46.949407 8323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b19b1751841758749a7575fe7a800" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41861 } }
I20260812 00:33:46.949957 8323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:46.950098 8323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326b19b1751841758749a7575fe7a800, State: Initialized, Role: FOLLOWER
I20260812 00:33:46.950891 8323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b19b1751841758749a7575fe7a800" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41861 } }
I20260812 00:33:46.951231 8323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:33:46.951403 8323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:33:46.951604 8323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:46.955240 8323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b19b1751841758749a7575fe7a800" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41861 } }
Aug 12 00:33:46 dist-test-slave-gjw7 krb5kdc[8162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:46.955966 8323 leader_election.cc:302] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 326b19b1751841758749a7575fe7a800; no voters:
I20260812 00:33:46.956785 8323 leader_election.cc:288] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:33:46.956898 8331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:33:46.958312 8331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [term 1 LEADER]: Becoming Leader. State: Replica: 326b19b1751841758749a7575fe7a800, State: Running, Role: LEADER
I20260812 00:33:46.959182 8331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b19b1751841758749a7575fe7a800" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41861 } }
I20260812 00:33:46.959657 8323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:33:46.962621 8332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "326b19b1751841758749a7575fe7a800" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b19b1751841758749a7575fe7a800" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41861 } } }
I20260812 00:33:46.963174 8332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [sys.catalog]: This master's current role is: LEADER
I20260812 00:33:46.962621 8334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 326b19b1751841758749a7575fe7a800. Latest consensus state: current_term: 1 leader_uuid: "326b19b1751841758749a7575fe7a800" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326b19b1751841758749a7575fe7a800" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41861 } } }
I20260812 00:33:46.963335 8334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800 [sys.catalog]: This master's current role is: LEADER
I20260812 00:33:46.972350 8327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:46.927543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58024 (local address 127.7.244.126:41861)
0812 00:33:46.929872 (+ 2329us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:46.929888 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:46.929946 (+ 58us) server_negotiation.cc:407] Connection header received
0812 00:33:46.939149 (+ 9203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:46.939178 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:46.939658 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:46.940051 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:46.944491 (+ 4440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:46.946030 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:46.948723 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:46.949559 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:46.949719 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:46.958052 (+ 8333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:46.958069 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:46.958093 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:46.958166 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:46.963620 (+ 5454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:46.965426 (+ 1806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:46.965438 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:46.965449 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:46.965782 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:46.966695 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:46.966701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:46.966704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:46.967930 (+ 1226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:46.968472 (+ 542us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:46.969727 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0812 00:33:46.970289 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2062,"thread_start_us":1772,"threads_started":1}
I20260812 00:33:46.980548 8345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:33:46.987516 8345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:33:46.998502 8345 catalog_manager.cc:1347] Generated new cluster ID: 13100997db8d4ab9ac79857295ca2176
I20260812 00:33:46.998646 8345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:33:47.012022 8345 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:33:47.014222 8345 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:33:47.025879 8345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 326b19b1751841758749a7575fe7a800: Generated new TSK 0
I20260812 00:33:47.026937 8345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:33:47.079520 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41861
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:47.477478 8376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:47.477811 8376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:47.477950 8376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:47.486027 8376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:47.486187 8376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:47.486344 8376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:33:47.500958 8376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:47.503008 8376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:47.504676 8376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:47.520025 8385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:47.520057 8383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:47.520112 8382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:47.521224 8376 server_base.cc:1034] running on GCE node
I20260812 00:33:47.521984 8376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:47.523013 8376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:47.524241 8376 hybrid_clock.cc:648] HybridClock initialized: now 1786494827524184 us; error 47 us; skew 500 ppm
I20260812 00:33:47.524590 8376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:47 dist-test-slave-gjw7 krb5kdc[8162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494827, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:47.528224 8376 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:33:47.529994 8376 webserver.cc:460] Webserver started at http://127.7.244.65:42773/ using document root <none> and password file <none>
I20260812 00:33:47.530762 8376 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:47.530910 8376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:47.531260 8376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:47.534719 8376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "e976ca1c2bd349c3af2d67a1b473cca9"
format_stamp: "Formatted at 2026-08-12 00:33:47 on dist-test-slave-gjw7"
server_key: "641f9655d8e73924d269d493bdd2fa2e"
server_key_iv: "c67f76c7de1bf7dbd7d238d07e60e879"
server_key_version: "encryptionkey@0"
I20260812 00:33:47.535640 8376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "e976ca1c2bd349c3af2d67a1b473cca9"
format_stamp: "Formatted at 2026-08-12 00:33:47 on dist-test-slave-gjw7"
server_key: "641f9655d8e73924d269d493bdd2fa2e"
server_key_iv: "c67f76c7de1bf7dbd7d238d07e60e879"
server_key_version: "encryptionkey@0"
I20260812 00:33:47.542054 8376 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:33:47.546401 8403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:47.548008 8376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.005s
I20260812 00:33:47.548218 8376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "e976ca1c2bd349c3af2d67a1b473cca9"
format_stamp: "Formatted at 2026-08-12 00:33:47 on dist-test-slave-gjw7"
server_key: "641f9655d8e73924d269d493bdd2fa2e"
server_key_iv: "c67f76c7de1bf7dbd7d238d07e60e879"
server_key_version: "encryptionkey@0"
I20260812 00:33:47.548441 8376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:47.580948 8376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:47.586036 8376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:47.586370 8376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:47.587558 8376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:33:47.589188 8376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:47.589282 8376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:47.589439 8376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:47.589495 8376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:47.623404 8376 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:37999
I20260812 00:33:47.623452 8625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:37999 every 8 connection(s)
I20260812 00:33:47.624934 8376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:33:47.635150 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 8376
I20260812 00:33:47.635438 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:33:47.636258 8145 external_mini_cluster.cc:1428] Setting key 4e35bc7ff2cd130ef843feb997f8d004
Aug 12 00:33:47 dist-test-slave-gjw7 krb5kdc[8162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494827, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:47.649003 8630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:47.627983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:33575 (local address 127.7.244.126:41861)
0812 00:33:47.628477 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:47.628483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:47.629609 (+ 1126us) server_negotiation.cc:407] Connection header received
0812 00:33:47.631587 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:47.631592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:47.631669 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:47.631783 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:47.633469 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:47.634199 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:47.636704 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:47.637262 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:47.637427 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:47.641865 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:47.641873 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:47.641877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:47.641925 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:47.645012 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:47.646084 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:47.646090 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:47.646092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:47.646253 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:47.646821 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:47.646826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:47.646828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:47.647185 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:47.647346 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:47.648258 (+ 912us) server_negotiation.cc:299] Negotiation successful
0812 00:33:47.648558 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":239,"threads_started":1}
I20260812 00:33:47.650182 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:47.628654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41861 (local address 127.7.244.65:33575)
0812 00:33:47.629443 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:47.629489 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:47.631281 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:47.632022 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:47.632032 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:47.632617 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:47.633260 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:47.633280 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:47.634348 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:47.634353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:47.636507 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:47.636529 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:47.637605 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:47.637614 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:47.637863 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:47.638611 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:47.638638 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:47.641453 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:47.645259 (+ 3806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:47.645271 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:47.645316 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:47.645902 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:47.646393 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:47.646400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:47.646404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:47.646666 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:47.647341 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:47.647353 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:47.647780 (+ 427us) client_negotiation.cc:769] Sending connection context
0812 00:33:47.648081 (+ 301us) client_negotiation.cc:240] Negotiation successful
0812 00:33:47.648714 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":301,"threads_started":1}
I20260812 00:33:47.652164 8627 heartbeater.cc:347] Connected to a master server at 127.7.244.126:41861
I20260812 00:33:47.652510 8627 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:47.653453 8627 heartbeater.cc:511] Master 127.7.244.126:41861 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:33:47.655747 8256 ts_manager.cc:194] Registered new tserver with Master: e976ca1c2bd349c3af2d67a1b473cca9 (127.7.244.65:37999)
I20260812 00:33:47.657310 8256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:33575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:33:47.685801 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41861
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:48.045887 8636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:48.046248 8636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:48.046392 8636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:48.054795 8636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:48.054970 8636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:48.055168 8636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:33:48.066388 8636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:48.067829 8636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:48.069795 8636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:48.084367 8650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:48.084429 8651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:48.084406 8654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:48.085350 8636 server_base.cc:1034] running on GCE node
I20260812 00:33:48.086084 8636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:48.087101 8636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:48.088336 8636 hybrid_clock.cc:648] HybridClock initialized: now 1786494828088285 us; error 39 us; skew 500 ppm
I20260812 00:33:48.088658 8636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:48.092104 8636 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:33:48.093732 8636 webserver.cc:460] Webserver started at http://127.7.244.66:38851/ using document root <none> and password file <none>
I20260812 00:33:48.094488 8636 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:48.094610 8636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:48.094947 8636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:48.098212 8636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "b97a78a5ed7846709f44d875f3093b2c"
format_stamp: "Formatted at 2026-08-12 00:33:48 on dist-test-slave-gjw7"
server_key: "3e7a48dac669b7ed2a81a13a661687e6"
server_key_iv: "a2d3ac8fb410dd4e9a2f17931970680e"
server_key_version: "encryptionkey@0"
I20260812 00:33:48.099104 8636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "b97a78a5ed7846709f44d875f3093b2c"
format_stamp: "Formatted at 2026-08-12 00:33:48 on dist-test-slave-gjw7"
server_key: "3e7a48dac669b7ed2a81a13a661687e6"
server_key_iv: "a2d3ac8fb410dd4e9a2f17931970680e"
server_key_version: "encryptionkey@0"
I20260812 00:33:48.105234 8636 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:33:48.109475 8665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:48.111040 8636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:33:48.111245 8636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "b97a78a5ed7846709f44d875f3093b2c"
format_stamp: "Formatted at 2026-08-12 00:33:48 on dist-test-slave-gjw7"
server_key: "3e7a48dac669b7ed2a81a13a661687e6"
server_key_iv: "a2d3ac8fb410dd4e9a2f17931970680e"
server_key_version: "encryptionkey@0"
I20260812 00:33:48.111498 8636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:48.139561 8636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:48.144595 8636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:48.144910 8636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:48.146113 8636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:33:48.148082 8636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:48.148182 8636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:48.148337 8636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:48.148404 8636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:48.182973 8636 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:41683
I20260812 00:33:48.183032 8892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:41683 every 8 connection(s)
I20260812 00:33:48.184360 8636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:33:48.187819 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 8636
I20260812 00:33:48.188057 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:33:48.188500 8145 external_mini_cluster.cc:1428] Setting key 145062f0ec439dc700ab8b104c3cadcc
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:48.208573 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:48.187507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:34865 (local address 127.7.244.126:41861)
0812 00:33:48.188090 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:48.188096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:48.189187 (+ 1091us) server_negotiation.cc:407] Connection header received
0812 00:33:48.191247 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:48.191253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:48.191343 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:48.191465 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:48.193749 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.194784 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.196653 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.197356 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.197501 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:48.201917 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:48.201923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:48.201927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:48.201968 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:48.204750 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.205918 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.205922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.205924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.206072 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.206641 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.206644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.206646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.206952 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:48.207067 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:48.208091 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0812 00:33:48.208284 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":293,"threads_started":1}
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
I20260812 00:33:48.209887 8900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:48.188059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41861 (local address 127.7.244.66:34865)
0812 00:33:48.188976 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:48.189027 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:48.190923 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:48.191686 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:48.191695 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:48.192525 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:48.193490 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:48.193522 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.195013 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.195018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:48.196443 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:48.196462 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.197601 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.197607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:48.197838 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:48.198619 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:48.198651 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:48.201612 (+ 2961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:48.204970 (+ 3358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:48.204982 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:48.205003 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:48.205764 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:48.206196 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:48.206201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:48.206205 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:48.206513 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:48.207101 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:48.207111 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:48.207639 (+ 528us) client_negotiation.cc:769] Sending connection context
0812 00:33:48.208011 (+ 372us) client_negotiation.cc:240] Negotiation successful
0812 00:33:48.208634 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":357,"threads_started":1}
I20260812 00:33:48.211494 8894 heartbeater.cc:347] Connected to a master server at 127.7.244.126:41861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
I20260812 00:33:48.211843 8894 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:48.212791 8894 heartbeater.cc:511] Master 127.7.244.126:41861 requested a full tablet report, sending...
I20260812 00:33:48.214798 8256 ts_manager.cc:194] Registered new tserver with Master: b97a78a5ed7846709f44d875f3093b2c (127.7.244.66:41683)
I20260812 00:33:48.215752 8256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:34865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:33:48.236910 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41861
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:41913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:48.592535 8914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:48.592854 8914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:48.592983 8914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:48.600695 8914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:48.600831 8914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:48.600991 8914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:33:48.611362 8914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41861
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:48.612792 8914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:48.614246 8914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:48.629767 8923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:48.630002 8927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:48.630254 8922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:48.631007 8914 server_base.cc:1034] running on GCE node
I20260812 00:33:48.631611 8914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:48.632565 8914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:48.633775 8914 hybrid_clock.cc:648] HybridClock initialized: now 1786494828633732 us; error 37 us; skew 500 ppm
I20260812 00:33:48.634091 8914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:48.637328 8914 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:33:48.638933 8914 webserver.cc:460] Webserver started at http://127.7.244.67:42387/ using document root <none> and password file <none>
I20260812 00:33:48.639667 8914 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:48.639806 8914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:48.640132 8914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:48.643267 8914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "9b141ff743354cc0b6a2094d9923694d"
format_stamp: "Formatted at 2026-08-12 00:33:48 on dist-test-slave-gjw7"
server_key: "a5256336d7bcb373ec28d2e83115a023"
server_key_iv: "60ca4a67e90baf63eac4139f737583a5"
server_key_version: "encryptionkey@0"
I20260812 00:33:48.644085 8914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "9b141ff743354cc0b6a2094d9923694d"
format_stamp: "Formatted at 2026-08-12 00:33:48 on dist-test-slave-gjw7"
server_key: "a5256336d7bcb373ec28d2e83115a023"
server_key_iv: "60ca4a67e90baf63eac4139f737583a5"
server_key_version: "encryptionkey@0"
I20260812 00:33:48.649967 8914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 00:33:48.653954 8939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:48.655448 8914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:33:48.655642 8914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "9b141ff743354cc0b6a2094d9923694d"
format_stamp: "Formatted at 2026-08-12 00:33:48 on dist-test-slave-gjw7"
server_key: "a5256336d7bcb373ec28d2e83115a023"
server_key_iv: "60ca4a67e90baf63eac4139f737583a5"
server_key_version: "encryptionkey@0"
I20260812 00:33:48.655879 8914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:48.660662 8627 heartbeater.cc:503] Master 127.7.244.126:41861 was elected leader, sending a full tablet report...
I20260812 00:33:48.670379 8914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:48.675256 8914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:48.675575 8914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:48.676699 8914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:33:48.678376 8914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:48.678465 8914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:48.678598 8914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:48.678659 8914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:48.711169 8914 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:40129
I20260812 00:33:48.711220 9172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:40129 every 8 connection(s)
I20260812 00:33:48.712498 8914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:33:48.717869 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 8914
I20260812 00:33:48.718094 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:33:48.718672 8145 external_mini_cluster.cc:1428] Setting key 8f0f491cfd969959c602f8c21b3f8a09
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:48.735831 9177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:48.715267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:56415 (local address 127.7.244.126:41861)
0812 00:33:48.715761 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:48.715766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:48.716960 (+ 1194us) server_negotiation.cc:407] Connection header received
0812 00:33:48.719628 (+ 2668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:48.719634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:48.719728 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:48.719855 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:48.721960 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.723026 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.724670 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.725117 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.725216 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:48.728947 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:48.728953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:48.728957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:48.728998 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:48.731929 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.732832 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.732836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.732838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.732979 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.733670 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.733676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.733679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.734034 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:48.734201 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:48.735206 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0812 00:33:48.735454 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":216,"threads_started":1}
I20260812 00:33:48.736845 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:48.715914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41861 (local address 127.7.244.67:56415)
0812 00:33:48.716784 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:48.716835 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:48.719269 (+ 2434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:48.720151 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:48.720160 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:48.720964 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:48.721780 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:48.721802 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.723185 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.723189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:48.724521 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:48.724537 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.725417 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.725425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:48.725643 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:48.726281 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:48.726305 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:48.728678 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:48.732144 (+ 3466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:48.732153 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:48.732170 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:48.732711 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:48.733135 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:48.733140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:48.733143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:48.733526 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:48.734207 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:48.734217 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:48.734748 (+ 531us) client_negotiation.cc:769] Sending connection context
0812 00:33:48.735064 (+ 316us) client_negotiation.cc:240] Negotiation successful
0812 00:33:48.735634 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":350,"threads_started":1}
I20260812 00:33:48.738294 9174 heartbeater.cc:347] Connected to a master server at 127.7.244.126:41861
I20260812 00:33:48.738606 9174 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:48.739341 9174 heartbeater.cc:511] Master 127.7.244.126:41861 requested a full tablet report, sending...
I20260812 00:33:48.740844 8256 ts_manager.cc:194] Registered new tserver with Master: 9b141ff743354cc0b6a2094d9923694d (127.7.244.67:40129)
I20260812 00:33:48.741904 8256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:56415
I20260812 00:33:48.748432 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:33:48.800278 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:48.782522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.7.244.65:37999)
0812 00:33:48.783398 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:48.783406 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:48.783436 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:33:48.783503 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:48.783510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:48.783750 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:48.783897 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:48.785879 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.786997 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.788560 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.789198 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.789395 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:48.793847 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:48.793865 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:48.793883 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:48.793926 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:48.797016 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.797935 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.797943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.797946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.798099 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.798589 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.798593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.798595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.799014 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:48.799333 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:48.799626 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 00:33:48.799861 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":552,"threads_started":1}
W20260812 00:33:48.801672 8455 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44492
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:33:48.850260 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:48.835391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44498 (local address 127.7.244.65:37999)
0812 00:33:48.835572 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:48.835577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:48.835956 (+ 379us) server_negotiation.cc:407] Connection header received
0812 00:33:48.836173 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:48.836178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:48.836317 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:48.836409 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:48.837867 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.838657 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.840024 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.840486 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.840600 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:48.844765 (+ 4165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:48.844771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:48.844775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:48.844817 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:48.847729 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.848481 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.848486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.848489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.848646 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.848960 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.848963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.848965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.849245 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:48.849501 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:48.849811 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 00:33:48.850011 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 00:33:48.851955 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 8376
I20260812 00:33:48.866214 9190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:48.853202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.7.244.65:37999)
0812 00:33:48.853487 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:48.853493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:48.853735 (+ 242us) server_negotiation.cc:407] Connection header received
0812 00:33:48.853907 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:48.853911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:48.853975 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:48.854061 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:48.855356 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.856110 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.857516 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.857965 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.858085 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:48.860676 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:48.860681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:48.860685 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:48.860724 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:48.863256 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.864018 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.864023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.864026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.864182 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.864632 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.864637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.864639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.864946 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:48.865190 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:48.865596 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 00:33:48.865827 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 00:33:48.867053 8617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:48.958403 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 8376
I20260812 00:33:48.979322 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 8636
Aug 12 00:33:48 dist-test-slave-gjw7 krb5kdc[8162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:33:48.996726 9203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:48.980230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.7.244.66:41683)
0812 00:33:48.980836 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:48.980843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:48.980868 (+ 25us) server_negotiation.cc:407] Connection header received
0812 00:33:48.980931 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:48.980938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:48.981156 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:48.981340 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:48.982612 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.983727 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.985058 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:48.985855 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:48.985997 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:48.990492 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:48.990512 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:48.990529 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:48.990577 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:48.993503 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.994325 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.994333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.994336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.994492 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:48.994944 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:48.994949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:48.994952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:48.995344 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:48.995680 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:48.996031 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 00:33:48.996338 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":323,"threads_started":1}
I20260812 00:33:48.998209 8885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:49.081326 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 8636
I20260812 00:33:49.106582 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 8914
Aug 12 00:33:49 dist-test-slave-gjw7 krb5kdc[8162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:33:49.123687 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:49.107667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34154 (local address 127.7.244.67:40129)
0812 00:33:49.108318 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:49.108325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:49.108355 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:33:49.108421 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:49.108428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:49.108645 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:49.108777 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:49.110173 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:49.111231 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:49.112502 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:49.113102 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:49.113207 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:49.117650 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:49.117667 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:49.117685 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:49.117727 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:49.120600 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:49.121489 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:49.121496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:49.121499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:49.121644 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:49.122092 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:49.122095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:49.122097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:49.122463 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:49.122720 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:49.123011 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 00:33:49.123242 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":334,"threads_started":1}
I20260812 00:33:49.125141 9165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:49.209151 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 8914
I20260812 00:33:49.231114 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 8194
I20260812 00:33:49.232406 8314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:49.336409 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 8194
2026-08-12T00:33:49Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4474 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:33:49 dist-test-slave-gjw7 krb5kdc[9215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:33:49 dist-test-slave-gjw7 krb5kdc[9215](info): set up 2 sockets
Aug 12 00:33:49 dist-test-slave-gjw7 krb5kdc[9215](info): commencing operation
krb5kdc: starting...
W20260812 00:33:50.636217 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:33:50 dist-test-slave-gjw7 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:33:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:33:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:33:50.849627 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41031
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:38241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:41031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:51.195056 9243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:51.195416 9243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:51.195720 9243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:51.203814 9243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:33:51.203991 9243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:51.204104 9243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:51.204208 9243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:33:51.204278 9243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:33:51.214926 9243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:38241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:41031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:51.216375 9243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:51.217809 9243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:51.230036 9253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:51.230116 9251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:51.229955 9250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:51.231184 9243 server_base.cc:1034] running on GCE node
I20260812 00:33:51.232091 9243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:51.233826 9243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:51.235074 9243 hybrid_clock.cc:648] HybridClock initialized: now 1786494831235016 us; error 56 us; skew 500 ppm
I20260812 00:33:51.235483 9243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:51 dist-test-slave-gjw7 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494831, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:51.243196 9243 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:33:51.245078 9243 webserver.cc:460] Webserver started at http://127.7.244.126:40899/ using document root <none> and password file <none>
I20260812 00:33:51.245929 9243 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:51.246071 9243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:51.246423 9243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:51.249902 9243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "4de8b7187978460ea701374ae7e28bb7"
format_stamp: "Formatted at 2026-08-12 00:33:51 on dist-test-slave-gjw7"
server_key: "5f8736a542da2cc306f69525e4b4cdbb"
server_key_iv: "6fa3f020c4b1a573c698171ce5f7f892"
server_key_version: "encryptionkey@0"
I20260812 00:33:51.250805 9243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "4de8b7187978460ea701374ae7e28bb7"
format_stamp: "Formatted at 2026-08-12 00:33:51 on dist-test-slave-gjw7"
server_key: "5f8736a542da2cc306f69525e4b4cdbb"
server_key_iv: "6fa3f020c4b1a573c698171ce5f7f892"
server_key_version: "encryptionkey@0"
I20260812 00:33:51.256971 9243 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:33:51.261152 9269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:51.262862 9243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:33:51.263083 9243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "4de8b7187978460ea701374ae7e28bb7"
format_stamp: "Formatted at 2026-08-12 00:33:51 on dist-test-slave-gjw7"
server_key: "5f8736a542da2cc306f69525e4b4cdbb"
server_key_iv: "6fa3f020c4b1a573c698171ce5f7f892"
server_key_version: "encryptionkey@0"
I20260812 00:33:51.263319 9243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:51.283128 9243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:51.287686 9243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:51.287993 9243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:51.306967 9243 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:41031
I20260812 00:33:51.306986 9378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:41031 every 8 connection(s)
I20260812 00:33:51.308491 9243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:33:51.311003 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 9243
I20260812 00:33:51.311231 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:33:51.311872 8145 external_mini_cluster.cc:1428] Setting key 75ad1c8f68f006e92cdcbf0fce9ee791
I20260812 00:33:51.312824 9380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 00:33:51 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:51.328455 9380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7: Bootstrap starting.
I20260812 00:33:51.332410 9380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:51.333863 9380 log.cc:824] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7: Log is configured to *not* fsync() on all Append() calls
I20260812 00:33:51.337177 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:51.313925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.7.244.126:41031)
0812 00:33:51.314719 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:51.314732 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:51.314774 (+ 42us) server_negotiation.cc:407] Connection header received
0812 00:33:51.316436 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:51.316462 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:51.316914 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:51.317313 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:51.319297 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:51.320741 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:51.322560 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:51.323312 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:51.323504 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:51.328860 (+ 5356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:51.328871 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:51.328890 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:51.328940 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:51.332314 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:51.333385 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:51.333393 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:51.333401 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:51.333594 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:51.334072 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:51.334076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:51.334079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:51.334903 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:51.335234 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:51.335646 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 00:33:51.336089 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":342,"threads_started":1}
I20260812 00:33:51.337567 9380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7: No bootstrap required, opened a new log
I20260812 00:33:51.345372 9380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de8b7187978460ea701374ae7e28bb7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41031 } }
I20260812 00:33:51.345821 9380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:51.345935 9380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4de8b7187978460ea701374ae7e28bb7, State: Initialized, Role: FOLLOWER
I20260812 00:33:51.346587 9380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de8b7187978460ea701374ae7e28bb7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41031 } }
I20260812 00:33:51.346820 9380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:33:51.346953 9380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:33:51.347131 9380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:51.349738 9380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de8b7187978460ea701374ae7e28bb7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41031 } }
I20260812 00:33:51.350328 9380 leader_election.cc:302] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4de8b7187978460ea701374ae7e28bb7; no voters:
I20260812 00:33:51.350955 9380 leader_election.cc:288] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:33:51.351123 9387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:33:51.352253 9387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [term 1 LEADER]: Becoming Leader. State: Replica: 4de8b7187978460ea701374ae7e28bb7, State: Running, Role: LEADER
I20260812 00:33:51.352936 9387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de8b7187978460ea701374ae7e28bb7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41031 } }
I20260812 00:33:51.353312 9380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:33:51.355979 9390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4de8b7187978460ea701374ae7e28bb7. Latest consensus state: current_term: 1 leader_uuid: "4de8b7187978460ea701374ae7e28bb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de8b7187978460ea701374ae7e28bb7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41031 } } }
I20260812 00:33:51.356316 9389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4de8b7187978460ea701374ae7e28bb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4de8b7187978460ea701374ae7e28bb7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41031 } } }
I20260812 00:33:51.356580 9390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:33:51.356657 9389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:33:51.360780 9403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:33:51.367067 9403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:33:51.377899 9403 catalog_manager.cc:1347] Generated new cluster ID: 13960c468f9546f8b7761eba6a425fc4
I20260812 00:33:51.378052 9403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:33:51.397567 9403 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:33:51.399019 9403 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:33:51.407630 9403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4de8b7187978460ea701374ae7e28bb7: Generated new TSK 0
I20260812 00:33:51.408643 9403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:33:51.469764 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41031
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:38241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:51.822564 9429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:51.822890 9429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:51.823024 9429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:51.830758 9429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:51.830890 9429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:51.831049 9429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:33:51.841897 9429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:38241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:51.843317 9429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:51.844784 9429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:51.858913 9446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:51.858917 9447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:51.859021 9451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:51.860137 9429 server_base.cc:1034] running on GCE node
I20260812 00:33:51.860824 9429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:51.861913 9429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:51.863195 9429 hybrid_clock.cc:648] HybridClock initialized: now 1786494831863100 us; error 41 us; skew 500 ppm
I20260812 00:33:51.863598 9429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:51 dist-test-slave-gjw7 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494831, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:51.867357 9429 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:33:51.869197 9429 webserver.cc:460] Webserver started at http://127.7.244.65:34057/ using document root <none> and password file <none>
I20260812 00:33:51.870025 9429 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:51.870149 9429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:51.870479 9429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:51.873957 9429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "696d1c077d654d84a58a2c493380da6d"
format_stamp: "Formatted at 2026-08-12 00:33:51 on dist-test-slave-gjw7"
server_key: "d6998e567bd76e191dd5405b2e631899"
server_key_iv: "097782514fa99992582120768900bb80"
server_key_version: "encryptionkey@0"
I20260812 00:33:51.874975 9429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "696d1c077d654d84a58a2c493380da6d"
format_stamp: "Formatted at 2026-08-12 00:33:51 on dist-test-slave-gjw7"
server_key: "d6998e567bd76e191dd5405b2e631899"
server_key_iv: "097782514fa99992582120768900bb80"
server_key_version: "encryptionkey@0"
I20260812 00:33:51.881984 9429 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260812 00:33:51.887463 9463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:51.889196 9429 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 00:33:51.889457 9429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "696d1c077d654d84a58a2c493380da6d"
format_stamp: "Formatted at 2026-08-12 00:33:51 on dist-test-slave-gjw7"
server_key: "d6998e567bd76e191dd5405b2e631899"
server_key_iv: "097782514fa99992582120768900bb80"
server_key_version: "encryptionkey@0"
I20260812 00:33:51.889688 9429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:51.914212 9429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:51.919094 9429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:51.919373 9429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:51.920485 9429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:33:51.922012 9429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:51.922099 9429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:51.922196 9429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:51.922240 9429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:51.954819 9429 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:37493
I20260812 00:33:51.954909 9682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:37493 every 8 connection(s)
I20260812 00:33:51.956238 9429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:33:51.961668 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 9429
I20260812 00:33:51.961930 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:33:51.962764 8145 external_mini_cluster.cc:1428] Setting key fcb3a47c51fd443337ff6a71044932b3
Aug 12 00:33:51 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494831, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:51.980808 9687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:51.959074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:47023 (local address 127.7.244.126:41031)
0812 00:33:51.959559 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:51.959564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:51.960799 (+ 1235us) server_negotiation.cc:407] Connection header received
0812 00:33:51.963606 (+ 2807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:51.963613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:51.963714 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:51.963829 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:51.966125 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:51.966890 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:51.969398 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:51.969834 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:51.969974 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:51.974370 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:51.974377 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:51.974380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:51.974431 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:51.977116 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:51.978105 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:51.978110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:51.978113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:51.978265 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:51.978774 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:51.978779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:51.978781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:51.979146 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:51.979307 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:51.980166 (+ 859us) server_negotiation.cc:299] Negotiation successful
0812 00:33:51.980446 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":221,"threads_started":1}
I20260812 00:33:51.981948 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:51.959780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41031 (local address 127.7.244.65:47023)
0812 00:33:51.960607 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:51.960659 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:51.963172 (+ 2513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:51.964153 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:51.964163 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:51.964858 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:51.965854 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:51.965886 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:51.967101 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:51.967107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:51.969141 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:51.969161 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:51.970113 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:51.970121 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:51.970437 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:51.971288 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:51.971323 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:51.974053 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:51.977360 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:51.977372 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:51.977390 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:51.977935 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:51.978399 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:51.978404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:51.978407 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:51.978633 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:51.979306 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:51.979315 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:51.979722 (+ 407us) client_negotiation.cc:769] Sending connection context
0812 00:33:51.979999 (+ 277us) client_negotiation.cc:240] Negotiation successful
0812 00:33:51.980616 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":295,"threads_started":1}
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:33:51.983783 9683 heartbeater.cc:347] Connected to a master server at 127.7.244.126:41031
I20260812 00:33:51.984146 9683 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:51.984942 9683 heartbeater.cc:511] Master 127.7.244.126:41031 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
I20260812 00:33:51.987706 9310 ts_manager.cc:194] Registered new tserver with Master: 696d1c077d654d84a58a2c493380da6d (127.7.244.65:37493)
I20260812 00:33:51.989760 9310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:47023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:33:52.013316 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41031
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:38241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:52.364482 9693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:52.364805 9693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:52.364908 9693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:52.372551 9693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:52.372676 9693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:52.372793 9693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:33:52.383993 9693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:38241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:52.386056 9693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:52.388211 9693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:52.403853 9699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:52.403918 9705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:52.403896 9698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:52.404824 9693 server_base.cc:1034] running on GCE node
I20260812 00:33:52.405596 9693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:52.406679 9693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:52.407943 9693 hybrid_clock.cc:648] HybridClock initialized: now 1786494832407859 us; error 64 us; skew 500 ppm
I20260812 00:33:52.408296 9693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:52 dist-test-slave-gjw7 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:52.411767 9693 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:33:52.413468 9693 webserver.cc:460] Webserver started at http://127.7.244.66:34387/ using document root <none> and password file <none>
I20260812 00:33:52.414263 9693 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:52.414381 9693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:52.414672 9693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:52.417979 9693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "62490d56733a476e9a3b85ce3320ca16"
format_stamp: "Formatted at 2026-08-12 00:33:52 on dist-test-slave-gjw7"
server_key: "fe14f0e4a4a4f7cc7ad04af810c2bf4c"
server_key_iv: "0303c5b614e2f34228ec4b698e9f00e2"
server_key_version: "encryptionkey@0"
I20260812 00:33:52.418862 9693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "62490d56733a476e9a3b85ce3320ca16"
format_stamp: "Formatted at 2026-08-12 00:33:52 on dist-test-slave-gjw7"
server_key: "fe14f0e4a4a4f7cc7ad04af810c2bf4c"
server_key_iv: "0303c5b614e2f34228ec4b698e9f00e2"
server_key_version: "encryptionkey@0"
I20260812 00:33:52.424783 9693 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:33:52.428768 9714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:52.430405 9693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:33:52.430593 9693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "62490d56733a476e9a3b85ce3320ca16"
format_stamp: "Formatted at 2026-08-12 00:33:52 on dist-test-slave-gjw7"
server_key: "fe14f0e4a4a4f7cc7ad04af810c2bf4c"
server_key_iv: "0303c5b614e2f34228ec4b698e9f00e2"
server_key_version: "encryptionkey@0"
I20260812 00:33:52.430809 9693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:52.451459 9693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:52.456310 9693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:52.456610 9693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:52.457747 9693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:33:52.459380 9693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:52.459471 9693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:52.459589 9693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:52.459637 9693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:52.491835 9693 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:38447
I20260812 00:33:52.491915 9975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:38447 every 8 connection(s)
I20260812 00:33:52.493242 9693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:33:52.495208 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 9693
I20260812 00:33:52.495409 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:33:52.496011 8145 external_mini_cluster.cc:1428] Setting key d43edace8e8edde650fa60d23ae89566
Aug 12 00:33:52 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
I20260812 00:33:52.517393 9980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:52.496231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:52647 (local address 127.7.244.126:41031)
0812 00:33:52.496702 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:52.496707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:52.497946 (+ 1239us) server_negotiation.cc:407] Connection header received
0812 00:33:52.499917 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:52.499923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:52.500009 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:52.500128 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:52.501889 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:52.502899 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:52.505012 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:52.505533 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:52.505641 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:52.510375 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:52.510382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:52.510386 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:52.510436 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:52.513507 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:52.514645 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:52.514650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:52.514652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:52.514804 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:52.515374 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:52.515378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:52.515380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:52.515661 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:52.515778 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:52.516839 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0812 00:33:52.517044 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260812 00:33:52.518597 9981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:52.496912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41031 (local address 127.7.244.66:52647)
0812 00:33:52.497772 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:52.497829 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:52.499583 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:52.500355 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:52.500364 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:52.500953 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:52.501679 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:52.501703 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:52.503104 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:52.503109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:52.504796 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:52.504816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:52.505793 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:52.505800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:52.506123 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:52.506923 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:52.506961 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:52.509956 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:52.513717 (+ 3761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:52.513729 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:52.513753 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:52.514480 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:52.514952 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:52.514958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:52.514961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:52.515256 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:52.515798 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:52.515808 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:52.516366 (+ 558us) client_negotiation.cc:769] Sending connection context
0812 00:33:52.516724 (+ 358us) client_negotiation.cc:240] Negotiation successful
0812 00:33:52.517237 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":353,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
I20260812 00:33:52.520493 9976 heartbeater.cc:347] Connected to a master server at 127.7.244.126:41031
I20260812 00:33:52.520824 9976 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:52.521781 9976 heartbeater.cc:511] Master 127.7.244.126:41031 requested a full tablet report, sending...
I20260812 00:33:52.523459 9310 ts_manager.cc:194] Registered new tserver with Master: 62490d56733a476e9a3b85ce3320ca16 (127.7.244.66:38447)
I20260812 00:33:52.524537 9310 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:52647
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:33:52.545470 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41031
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:38241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:52.890071 9987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:52.890421 9987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:52.890566 9987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:52.898288 9987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:52.898454 9987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:52.898620 9987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:33:52.908963 9987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:38241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:41031
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:52.910416 9987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:52.911903 9987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:52.925828 9996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:52.925733 9995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:52.926529 9999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:52.926784 9987 server_base.cc:1034] running on GCE node
I20260812 00:33:52.927524 9987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:52.928586 9987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:52.929798 9987 hybrid_clock.cc:648] HybridClock initialized: now 1786494832929749 us; error 49 us; skew 500 ppm
I20260812 00:33:52.930135 9987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:52 dist-test-slave-gjw7 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:52.933533 9987 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:33:52.935350 9987 webserver.cc:460] Webserver started at http://127.7.244.67:45327/ using document root <none> and password file <none>
I20260812 00:33:52.936241 9987 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:52.936386 9987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:52.936743 9987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:52.940130 9987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "6c409a6d02d640bcbd767c0d974781e0"
format_stamp: "Formatted at 2026-08-12 00:33:52 on dist-test-slave-gjw7"
server_key: "7969482898b2aa6da65b9d1d39a89f63"
server_key_iv: "7b946bbad9aeba05301b649b25772506"
server_key_version: "encryptionkey@0"
I20260812 00:33:52.941075 9987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "6c409a6d02d640bcbd767c0d974781e0"
format_stamp: "Formatted at 2026-08-12 00:33:52 on dist-test-slave-gjw7"
server_key: "7969482898b2aa6da65b9d1d39a89f63"
server_key_iv: "7b946bbad9aeba05301b649b25772506"
server_key_version: "encryptionkey@0"
I20260812 00:33:52.947227 9987 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 00:33:52.951465 10014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:52.953099 9987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:33:52.953352 9987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "6c409a6d02d640bcbd767c0d974781e0"
format_stamp: "Formatted at 2026-08-12 00:33:52 on dist-test-slave-gjw7"
server_key: "7969482898b2aa6da65b9d1d39a89f63"
server_key_iv: "7b946bbad9aeba05301b649b25772506"
server_key_version: "encryptionkey@0"
I20260812 00:33:52.953583 9987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:52.986789 9987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:52.991475 9987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:52.991783 9987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:52.992854 9987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:33:52.993562 9683 heartbeater.cc:503] Master 127.7.244.126:41031 was elected leader, sending a full tablet report...
I20260812 00:33:52.994508 9987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:52.994596 9987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:52.994727 9987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:52.994791 9987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:53.027185 9987 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:39911
I20260812 00:33:53.027235 10258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:39911 every 8 connection(s)
I20260812 00:33:53.028661 9987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:33:53.038067 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 9987
I20260812 00:33:53.038297 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:33:53.038764 8145 external_mini_cluster.cc:1428] Setting key 53436202b29880478c71b7371382b549
Aug 12 00:33:53 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494832, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:53.051397 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.031453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:53731 (local address 127.7.244.126:41031)
0812 00:33:53.031985 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.031989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.032960 (+ 971us) server_negotiation.cc:407] Connection header received
0812 00:33:53.035130 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.035136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.035249 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.035372 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:53.037183 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.038229 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.040189 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.040608 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.040753 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.044979 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:53.044985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:53.044989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:53.045029 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:53.047904 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.048845 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.048850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.048852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.049016 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.049585 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.049589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.049591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.049931 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:53.050072 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.050841 (+ 769us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.051048 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":263,"threads_started":1}
I20260812 00:33:53.052361 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.031949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41031 (local address 127.7.244.67:53731)
0812 00:33:53.032773 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:53.032816 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:53.034722 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:53.035669 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:53.035679 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:53.036349 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:53.036981 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.037002 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.038528 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.038534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.040021 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.040041 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.040931 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.040941 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.041255 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.042027 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:53.042058 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:53.044695 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:53.048112 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:53.048122 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:53.048138 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:53.048678 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:53.049134 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:53.049137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:53.049139 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:53.049446 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:53.050039 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:53.050046 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:53.050439 (+ 393us) client_negotiation.cc:769] Sending connection context
0812 00:33:53.050700 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 00:33:53.051143 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":321,"threads_started":1}
I20260812 00:33:53.054090 10261 heartbeater.cc:347] Connected to a master server at 127.7.244.126:41031
I20260812 00:33:53.054462 10261 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:53.055426 10261 heartbeater.cc:511] Master 127.7.244.126:41031 requested a full tablet report, sending...
I20260812 00:33:53.057055 9306 ts_manager.cc:194] Registered new tserver with Master: 6c409a6d02d640bcbd767c0d974781e0 (127.7.244.67:39911)
I20260812 00:33:53.058087 9306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:53731
I20260812 00:33:53.067981 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:33:53.088315 10266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.074925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46060 (local address 127.7.244.126:41031)
0812 00:33:53.075190 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.075197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.075571 (+ 374us) server_negotiation.cc:407] Connection header received
0812 00:33:53.075739 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.075743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.075817 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.075911 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:53.077615 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.078414 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.079863 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.080309 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.080421 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.082972 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:53.082977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:53.082981 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:53.083030 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:53.085655 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.086387 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.086392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.086396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.086538 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.086862 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.086864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.086866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.087216 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:53.087409 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.087794 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.088011 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 00:33:53.096306 9306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:33:53.103608 9306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:33:53.143549 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.118624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.7.244.65:37493)
0812 00:33:53.127970 (+ 9346us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.127980 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.135268 (+ 7288us) server_negotiation.cc:407] Connection header received
0812 00:33:53.135641 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.135650 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.135873 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.136121 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:53.136885 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.138419 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.140711 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.142275 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.142418 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.142765 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.143028 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.143204 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9154,"thread_start_us":8939,"threads_started":1}
I20260812 00:33:53.143676 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.117919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:37493 (local address 127.0.0.1:59718)
0812 00:33:53.135115 (+ 17196us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:53.135143 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:53.135401 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:53.136302 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:53.136307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:53.136334 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:53.136698 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.136710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.138591 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.138594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.140470 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.140486 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.142513 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.142520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.142754 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.142817 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 00:33:53.142946 (+ 129us) client_negotiation.cc:240] Negotiation successful
0812 00:33:53.143174 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17028,"mutex_wait_us":77,"thread_start_us":15636,"threads_started":1}
I20260812 00:33:53.144426 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.117356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:38447 (local address 127.0.0.1:47118)
0812 00:33:53.133207 (+ 15851us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:53.133242 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:53.133442 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:53.135789 (+ 2347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:53.135794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:53.135826 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:53.136290 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.136302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.138205 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.138209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.141062 (+ 2853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.141076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.143451 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.143459 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.143682 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.143721 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 00:33:53.143816 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 00:33:53.143990 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15663,"mutex_wait_us":72,"thread_start_us":17636,"threads_started":1}
I20260812 00:33:53.144763 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.117956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.7.244.66:38447)
0812 00:33:53.135078 (+ 17122us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.135088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.135118 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:33:53.135218 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.135226 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.135555 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.135773 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:53.136507 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.138060 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.141278 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.142929 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.143054 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.143235 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.144251 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.144446 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16931,"thread_start_us":16794,"threads_started":1}
I20260812 00:33:53.145195 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.117540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49368 (local address 127.7.244.67:39911)
0812 00:33:53.134354 (+ 16814us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.134363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.134401 (+ 38us) server_negotiation.cc:407] Connection header received
0812 00:33:53.134516 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.134524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.134848 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.135158 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:53.136142 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.137979 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.141283 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.142874 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.143025 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.143260 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.144725 (+ 1465us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.144897 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16621,"thread_start_us":16458,"threads_started":1}
I20260812 00:33:53.145864 10289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.117113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:39911 (local address 127.0.0.1:49368)
0812 00:33:53.131955 (+ 14842us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:53.132024 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:53.132284 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:53.135387 (+ 3103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:53.135395 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:53.135424 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:53.135888 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.135912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.138188 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.138194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.141058 (+ 2864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.141076 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.143463 (+ 2387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.143468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.144135 (+ 667us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.144168 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 00:33:53.144261 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 00:33:53.144417 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14468,"mutex_wait_us":222,"thread_start_us":15437,"threads_started":1}
I20260812 00:33:53.148067 9556 tablet_service.cc:1467] Processing CreateTablet for tablet 0db17533559d4bfdad91f6c5bff7fcbc (DEFAULT_TABLE table=test-table [id=8e0d5010a4d74a1b9d72c86c431dacbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:53.149920 9556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db17533559d4bfdad91f6c5bff7fcbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:53.150445 10120 tablet_service.cc:1467] Processing CreateTablet for tablet 0db17533559d4bfdad91f6c5bff7fcbc (DEFAULT_TABLE table=test-table [id=8e0d5010a4d74a1b9d72c86c431dacbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:53.152189 10120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db17533559d4bfdad91f6c5bff7fcbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:53.152426 9822 tablet_service.cc:1467] Processing CreateTablet for tablet 0db17533559d4bfdad91f6c5bff7fcbc (DEFAULT_TABLE table=test-table [id=8e0d5010a4d74a1b9d72c86c431dacbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:53.154196 9822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db17533559d4bfdad91f6c5bff7fcbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:53.166157 10305 tablet_bootstrap.cc:492] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16: Bootstrap starting.
I20260812 00:33:53.170217 10307 tablet_bootstrap.cc:492] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0: Bootstrap starting.
I20260812 00:33:53.171829 10305 tablet_bootstrap.cc:654] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:53.172427 10306 tablet_bootstrap.cc:492] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d: Bootstrap starting.
I20260812 00:33:53.173463 10305 log.cc:824] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16: Log is configured to *not* fsync() on all Append() calls
I20260812 00:33:53.174531 10307 tablet_bootstrap.cc:654] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:53.175705 10306 tablet_bootstrap.cc:654] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:53.175868 10307 log.cc:824] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:33:53.177117 10306 log.cc:824] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:33:53.178450 10305 tablet_bootstrap.cc:492] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16: No bootstrap required, opened a new log
I20260812 00:33:53.178776 10305 ts_tablet_manager.cc:1397] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260812 00:33:53.179426 10307 tablet_bootstrap.cc:492] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0: No bootstrap required, opened a new log
I20260812 00:33:53.179741 10307 ts_tablet_manager.cc:1397] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260812 00:33:53.180557 10306 tablet_bootstrap.cc:492] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d: No bootstrap required, opened a new log
I20260812 00:33:53.180840 10306 ts_tablet_manager.cc:1397] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 00:33:53.186307 10305 raft_consensus.cc:348] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.186746 10305 raft_consensus.cc:374] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:53.186851 10305 raft_consensus.cc:729] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62490d56733a476e9a3b85ce3320ca16, State: Initialized, Role: FOLLOWER
I20260812 00:33:53.187727 10305 consensus_queue.cc:260] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.189842 9976 heartbeater.cc:503] Master 127.7.244.126:41031 was elected leader, sending a full tablet report...
I20260812 00:33:53.190505 10305 ts_tablet_manager.cc:1428] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260812 00:33:53.190769 10307 raft_consensus.cc:348] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.191287 10307 raft_consensus.cc:374] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:53.191401 10307 raft_consensus.cc:729] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c409a6d02d640bcbd767c0d974781e0, State: Initialized, Role: FOLLOWER
I20260812 00:33:53.192080 10307 consensus_queue.cc:260] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.192188 10306 raft_consensus.cc:348] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.192612 10306 raft_consensus.cc:374] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:53.192713 10306 raft_consensus.cc:729] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696d1c077d654d84a58a2c493380da6d, State: Initialized, Role: FOLLOWER
I20260812 00:33:53.193392 10306 consensus_queue.cc:260] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.195437 10306 ts_tablet_manager.cc:1428] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260812 00:33:53.196694 10261 heartbeater.cc:503] Master 127.7.244.126:41031 was elected leader, sending a full tablet report...
I20260812 00:33:53.197517 10307 ts_tablet_manager.cc:1428] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0: Time spent starting tablet: real 0.018s user 0.020s sys 0.000s
W20260812 00:33:53.208865 9684 tablet.cc:2369] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:33:53.245465 9978 tablet.cc:2369] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:33:53.280548 10264 tablet.cc:2369] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:33:53.282677 10321 raft_consensus.cc:482] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:33:53.282967 10321 raft_consensus.cc:504] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.283834 10322 raft_consensus.cc:482] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:33:53.284117 10322 raft_consensus.cc:504] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.284849 10321 leader_election.cc:288] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62490d56733a476e9a3b85ce3320ca16 (127.7.244.66:38447), 696d1c077d654d84a58a2c493380da6d (127.7.244.65:37493)
I20260812 00:33:53.286840 10322 leader_election.cc:288] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c409a6d02d640bcbd767c0d974781e0 (127.7.244.67:39911), 62490d56733a476e9a3b85ce3320ca16 (127.7.244.66:38447)
I20260812 00:33:53.293916 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.285131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:38447 (local address 127.7.244.67:40687)
0812 00:33:53.285625 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:53.285647 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:53.285790 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:53.287142 (+ 1352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:53.287146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:53.287175 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:53.287419 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.287435 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.288983 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.288989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.291160 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.291179 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.292962 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.292969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.293173 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.293212 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 00:33:53.293374 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 00:33:53.293544 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":33,"spinlock_wait_cycles":17024}
I20260812 00:33:53.294972 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.285435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:58621 (local address 127.7.244.65:37493)
0812 00:33:53.285647 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.285653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.286490 (+ 837us) server_negotiation.cc:407] Connection header received
0812 00:33:53.286767 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.286776 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.286852 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.286931 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:53.287770 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.288875 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.292386 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.293703 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.293912 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.294055 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.294476 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.294690 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":67}
I20260812 00:33:53.295047 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.285790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:40687 (local address 127.7.244.66:38447)
0812 00:33:53.286552 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.286558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.286575 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:33:53.286662 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.286668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.286746 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.286828 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:53.287645 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.288802 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.291355 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.292806 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.292953 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.293013 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.294596 (+ 1583us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.294760 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611}
I20260812 00:33:53.296241 9863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db17533559d4bfdad91f6c5bff7fcbc" candidate_uuid: "6c409a6d02d640bcbd767c0d974781e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62490d56733a476e9a3b85ce3320ca16" is_pre_election: true
I20260812 00:33:53.296957 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.286359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:40725 (local address 127.7.244.67:39911)
0812 00:33:53.287091 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.287097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.287605 (+ 508us) server_negotiation.cc:407] Connection header received
0812 00:33:53.287863 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.287868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.287949 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.288013 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:53.288924 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.290151 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.294897 (+ 4746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.296002 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.296150 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.296223 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.296567 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.296697 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595}
I20260812 00:33:53.297060 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.286218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:39911 (local address 127.7.244.65:40725)
0812 00:33:53.287509 (+ 1291us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:53.287526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:53.287703 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:53.288208 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:53.288212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:53.288241 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:53.288707 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.288718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.290279 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.290284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.294694 (+ 4410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.294714 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.296227 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.296234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.296396 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.296428 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 00:33:53.296516 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 00:33:53.296659 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"mutex_wait_us":2,"thread_start_us":1587,"threads_started":1}
I20260812 00:33:53.297022 9863 raft_consensus.cc:2454] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c409a6d02d640bcbd767c0d974781e0 in term 0.
I20260812 00:33:53.297942 10323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.285346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:37493 (local address 127.7.244.67:58621)
0812 00:33:53.286385 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:53.286406 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:53.286598 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:53.287128 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:53.287132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:53.287160 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:53.287593 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.287605 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.289091 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.289096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.292168 (+ 3072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.292185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.294087 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.294093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.294277 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.294312 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:33:53.295251 (+ 939us) client_negotiation.cc:240] Negotiation successful
0812 00:33:53.295421 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":1,"thread_start_us":2647,"threads_started":1}
I20260812 00:33:53.298231 10021 leader_election.cc:302] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62490d56733a476e9a3b85ce3320ca16, 6c409a6d02d640bcbd767c0d974781e0; no voters:
I20260812 00:33:53.298987 10321 raft_consensus.cc:2781] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:33:53.299180 10321 raft_consensus.cc:482] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:33:53.299293 10321 raft_consensus.cc:3037] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:53.299188 9588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db17533559d4bfdad91f6c5bff7fcbc" candidate_uuid: "6c409a6d02d640bcbd767c0d974781e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696d1c077d654d84a58a2c493380da6d" is_pre_election: true
I20260812 00:33:53.299780 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.286199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:38447 (local address 127.7.244.65:57985)
0812 00:33:53.286902 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:53.286920 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:53.287171 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:53.290414 (+ 3243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:53.290420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:53.290452 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:53.290907 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.290921 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.293428 (+ 2507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.293435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.296100 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:53.296117 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.298832 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.298838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:53.299041 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.299078 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 00:33:53.299193 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 00:33:53.299355 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":85,"thread_start_us":399,"threads_started":1}
I20260812 00:33:53.300221 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.287535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:57985 (local address 127.7.244.66:38447)
0812 00:33:53.290050 (+ 2515us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.290056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.290080 (+ 24us) server_negotiation.cc:407] Connection header received
0812 00:33:53.290140 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.290146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.290215 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.291054 (+ 839us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:53.291331 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.292780 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.296335 (+ 3555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.298315 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.298521 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.298596 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.299491 (+ 895us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.299725 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2361,"thread_start_us":2171,"threads_started":1}
I20260812 00:33:53.299800 9588 raft_consensus.cc:2454] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c409a6d02d640bcbd767c0d974781e0 in term 0.
I20260812 00:33:53.300261 10166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db17533559d4bfdad91f6c5bff7fcbc" candidate_uuid: "696d1c077d654d84a58a2c493380da6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c409a6d02d640bcbd767c0d974781e0" is_pre_election: true
I20260812 00:33:53.300976 9863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db17533559d4bfdad91f6c5bff7fcbc" candidate_uuid: "696d1c077d654d84a58a2c493380da6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62490d56733a476e9a3b85ce3320ca16" is_pre_election: true
I20260812 00:33:53.301203 9863 raft_consensus.cc:2454] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 696d1c077d654d84a58a2c493380da6d in term 0.
I20260812 00:33:53.301783 9465 leader_election.cc:302] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62490d56733a476e9a3b85ce3320ca16, 696d1c077d654d84a58a2c493380da6d; no voters:
I20260812 00:33:53.302232 10322 raft_consensus.cc:2781] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:33:53.302397 10322 raft_consensus.cc:482] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:33:53.302505 10322 raft_consensus.cc:3037] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:53.302670 10321 raft_consensus.cc:504] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.303140 10166 raft_consensus.cc:2379] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 696d1c077d654d84a58a2c493380da6d in current term 1: Already voted for candidate 6c409a6d02d640bcbd767c0d974781e0 in this term.
I20260812 00:33:53.303838 9863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db17533559d4bfdad91f6c5bff7fcbc" candidate_uuid: "6c409a6d02d640bcbd767c0d974781e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62490d56733a476e9a3b85ce3320ca16"
I20260812 00:33:53.304096 9863 raft_consensus.cc:3037] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:53.304116 9588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db17533559d4bfdad91f6c5bff7fcbc" candidate_uuid: "6c409a6d02d640bcbd767c0d974781e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696d1c077d654d84a58a2c493380da6d"
I20260812 00:33:53.304471 10321 leader_election.cc:288] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [CANDIDATE]: Term 1 election: Requested vote from peers 62490d56733a476e9a3b85ce3320ca16 (127.7.244.66:38447), 696d1c077d654d84a58a2c493380da6d (127.7.244.65:37493)
I20260812 00:33:53.305775 10322 raft_consensus.cc:504] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.306246 9588 raft_consensus.cc:2379] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c409a6d02d640bcbd767c0d974781e0 in current term 1: Already voted for candidate 696d1c077d654d84a58a2c493380da6d in this term.
I20260812 00:33:53.307044 9863 raft_consensus.cc:2454] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c409a6d02d640bcbd767c0d974781e0 in term 1.
I20260812 00:33:53.307564 10021 leader_election.cc:302] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62490d56733a476e9a3b85ce3320ca16, 6c409a6d02d640bcbd767c0d974781e0; no voters:
I20260812 00:33:53.307934 10321 raft_consensus.cc:2781] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:33:53.308439 10321 raft_consensus.cc:686] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [term 1 LEADER]: Becoming Leader. State: Replica: 6c409a6d02d640bcbd767c0d974781e0, State: Running, Role: LEADER
I20260812 00:33:53.308904 10321 consensus_queue.cc:237] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } }
I20260812 00:33:53.314077 10322 leader_election.cc:288] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [CANDIDATE]: Term 1 election: Requested vote from peers 6c409a6d02d640bcbd767c0d974781e0 (127.7.244.67:39911), 62490d56733a476e9a3b85ce3320ca16 (127.7.244.66:38447)
I20260812 00:33:53.314183 9863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db17533559d4bfdad91f6c5bff7fcbc" candidate_uuid: "696d1c077d654d84a58a2c493380da6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62490d56733a476e9a3b85ce3320ca16"
I20260812 00:33:53.314481 9863 raft_consensus.cc:2379] T 0db17533559d4bfdad91f6c5bff7fcbc P 62490d56733a476e9a3b85ce3320ca16 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 696d1c077d654d84a58a2c493380da6d in current term 1: Already voted for candidate 6c409a6d02d640bcbd767c0d974781e0 in this term.
I20260812 00:33:53.313855 9306 catalog_manager.cc:5495] T 0db17533559d4bfdad91f6c5bff7fcbc P 6c409a6d02d640bcbd767c0d974781e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c409a6d02d640bcbd767c0d974781e0 (127.7.244.67). New cstate: current_term: 1 leader_uuid: "6c409a6d02d640bcbd767c0d974781e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c409a6d02d640bcbd767c0d974781e0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39911 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62490d56733a476e9a3b85ce3320ca16" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "696d1c077d654d84a58a2c493380da6d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 37493 } health_report { overall_health: UNKNOWN } } }
I20260812 00:33:53.315723 10166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db17533559d4bfdad91f6c5bff7fcbc" candidate_uuid: "696d1c077d654d84a58a2c493380da6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c409a6d02d640bcbd767c0d974781e0"
I20260812 00:33:53.316937 9466 leader_election.cc:302] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 696d1c077d654d84a58a2c493380da6d; no voters: 62490d56733a476e9a3b85ce3320ca16, 6c409a6d02d640bcbd767c0d974781e0
I20260812 00:33:53.317286 10322 raft_consensus.cc:2726] T 0db17533559d4bfdad91f6c5bff7fcbc P 696d1c077d654d84a58a2c493380da6d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
Aug 12 00:33:53 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:33:53.347923 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.332169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.7.244.65:37493)
0812 00:33:53.332381 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.332388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.332888 (+ 500us) server_negotiation.cc:407] Connection header received
0812 00:33:53.333021 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.333024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.333104 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.333189 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:53.334843 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.335752 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.337057 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.337552 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.337663 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.341905 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:53.341924 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:53.341944 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:53.341986 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:53.344979 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.345828 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.345836 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.345838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.345975 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.346274 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.346277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.346279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.346688 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:53.346939 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.347332 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.347573 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 00:33:53 dist-test-slave-gjw7 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494833, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:33:53 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494833, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:33:53.401696 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.386827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.7.244.65:37493)
0812 00:33:53.387056 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.387064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.387411 (+ 347us) server_negotiation.cc:407] Connection header received
0812 00:33:53.387642 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.387648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.387726 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.387822 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:53.389263 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.390116 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.391472 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.391956 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.392063 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.396393 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:53.396399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:53.396403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:53.396448 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:53.399114 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.399974 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.399979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.399981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.400132 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.400511 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.400514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.400516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.400798 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:53.401014 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.401211 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.401449 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 00:33:53 dist-test-slave-gjw7 krb5kdc[9215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:33:53 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:33:53.452292 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.437485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59752 (local address 127.7.244.65:37493)
0812 00:33:53.437679 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.437686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.438054 (+ 368us) server_negotiation.cc:407] Connection header received
0812 00:33:53.438271 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.438275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.438349 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.438438 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:53.439926 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.440664 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.442039 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.442503 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.442626 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.446866 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:53.446872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:53.446876 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:53.446921 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:53.449626 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.450453 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.450458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.450460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.450619 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.451003 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.451006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.451008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.451333 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:53.451554 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.451800 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.452024 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 00:33:53.454241 9512 tablet_service.cc:2908] Rejecting scan request for tablet 0db17533559d4bfdad91f6c5bff7fcbc: Uninitialized: clean time has not yet been initialized
I20260812 00:33:53.458410 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 9429
I20260812 00:33:53.472211 10304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.459476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.7.244.65:37493)
0812 00:33:53.459642 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.459646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.459982 (+ 336us) server_negotiation.cc:407] Connection header received
0812 00:33:53.460141 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.460145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.460224 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.460306 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:53.461684 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.462407 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.463747 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.464213 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.464326 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.466820 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:53.466825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:53.466829 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:53.466868 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:53.469650 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.470415 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.470420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.470423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.470587 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.470937 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.470941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.470943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.471250 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:53.471467 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.471750 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.471951 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 00:33:53.472935 9675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:53.565724 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 9429
I20260812 00:33:53.590658 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 9693
Aug 12 00:33:53 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:33:53.606048 10329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.591455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47120 (local address 127.7.244.66:38447)
0812 00:33:53.591668 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.591674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.591695 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:33:53.591903 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.591909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.591978 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.592068 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:53.593446 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.594177 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.595525 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.595938 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.596052 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.600304 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:53.600322 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:53.600340 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:53.600388 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:53.603177 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.603886 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.603895 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.603898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.604036 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.604426 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.604430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.604432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.604841 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:53.605133 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.605485 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.605732 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:33:53.606825 9970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:53.695528 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 9693
I20260812 00:33:53.721040 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 9987
Aug 12 00:33:53 dist-test-slave-gjw7 krb5kdc[9215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:33:53.736722 10298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:53.721920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49382 (local address 127.7.244.67:39911)
0812 00:33:53.722106 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:53.722111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:53.722130 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:33:53.722239 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:53.722243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:53.722308 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:53.722393 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:53.723817 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.724586 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.726165 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:53.726577 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:53.726676 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:53.730921 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:53.730939 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:53.730956 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:53.731000 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:53.733900 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.734667 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.734676 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.734678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.734828 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:53.735261 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:53.735264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:53.735266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:53.735713 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:53.735960 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:53.736188 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 00:33:53.736370 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:33:53.737573 10252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:53.827795 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 9987
I20260812 00:33:53.847885 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 9243
I20260812 00:33:53.849159 9368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:53.961081 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 9243
2026-08-12T00:33:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4620 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:33:54 dist-test-slave-gjw7 krb5kdc[10372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:33:54 dist-test-slave-gjw7 krb5kdc[10372](info): set up 2 sockets
Aug 12 00:33:54 dist-test-slave-gjw7 krb5kdc[10372](info): commencing operation
krb5kdc: starting...
W20260812 00:33:55.241688 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:33:55 dist-test-slave-gjw7 krb5kdc[10372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:33:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:33:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:33:55.473219 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44849
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:44849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:55.822942 10398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:55.823299 10398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:55.823453 10398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:55.831028 10398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:33:55.831179 10398 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 00:33:55.831243 10398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:55.831337 10398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:55.831419 10398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:33:55.831512 10398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:33:55.842188 10398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:44849
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:55.843685 10398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:55.845041 10398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:55.857479 10412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:55.857419 10408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:55.857497 10405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:55.858323 10398 server_base.cc:1034] running on GCE node
I20260812 00:33:55.859200 10398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:55.860780 10398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:55.862061 10398 hybrid_clock.cc:648] HybridClock initialized: now 1786494835861973 us; error 45 us; skew 500 ppm
I20260812 00:33:55.862439 10398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:55 dist-test-slave-gjw7 krb5kdc[10372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494835, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:55.865875 10398 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:33:55.867533 10398 webserver.cc:460] Webserver started at http://127.7.244.126:40785/ using document root <none> and password file <none>
I20260812 00:33:55.868346 10398 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:55.868472 10398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:55.868839 10398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:55.872274 10398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "81550dfc70ad4409b041664e6ee81beb"
format_stamp: "Formatted at 2026-08-12 00:33:55 on dist-test-slave-gjw7"
server_key: "425ae77d4a81c562514536d2ccbe2afa"
server_key_iv: "e8b4a77f81096539fe59b589ef81e5fe"
server_key_version: "encryptionkey@0"
I20260812 00:33:55.873225 10398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "81550dfc70ad4409b041664e6ee81beb"
format_stamp: "Formatted at 2026-08-12 00:33:55 on dist-test-slave-gjw7"
server_key: "425ae77d4a81c562514536d2ccbe2afa"
server_key_iv: "e8b4a77f81096539fe59b589ef81e5fe"
server_key_version: "encryptionkey@0"
I20260812 00:33:55.880196 10398 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260812 00:33:55.885684 10429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:55.887918 10398 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:33:55.888134 10398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "81550dfc70ad4409b041664e6ee81beb"
format_stamp: "Formatted at 2026-08-12 00:33:55 on dist-test-slave-gjw7"
server_key: "425ae77d4a81c562514536d2ccbe2afa"
server_key_iv: "e8b4a77f81096539fe59b589ef81e5fe"
server_key_version: "encryptionkey@0"
I20260812 00:33:55.888361 10398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:55.907734 10398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:55.912492 10398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:55.912744 10398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:55.934924 10398 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:44849
I20260812 00:33:55.935009 10565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:44849 every 8 connection(s)
I20260812 00:33:55.936403 10398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:33:55.940467 10567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:55.945690 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 10398
I20260812 00:33:55.945967 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:33:55.946780 8145 external_mini_cluster.cc:1428] Setting key 6870cd5760abef487b6f1cf8e69400d0
I20260812 00:33:55.953143 10567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb: Bootstrap starting.
I20260812 00:33:55.957567 10567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:55.958895 10567 log.cc:824] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:33:55 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:55.962929 10567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb: No bootstrap required, opened a new log
I20260812 00:33:55.971923 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:55.948975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39896 (local address 127.7.244.126:44849)
0812 00:33:55.949785 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:55.949798 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:55.949847 (+ 49us) server_negotiation.cc:407] Connection header received
0812 00:33:55.951676 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:55.951703 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:55.952163 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:55.952572 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:55.954579 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:55.956030 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:55.957876 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:55.958656 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:55.958812 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:55.963921 (+ 5109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:55.963932 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:55.963952 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:55.964001 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:55.967287 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:55.968244 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:55.968252 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:55.968261 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:55.968463 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:55.968867 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:55.968871 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:55.968873 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:55.969594 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:55.969833 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:55.970329 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 00:33:55.970734 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":367,"threads_started":1}
I20260812 00:33:55.972354 10567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81550dfc70ad4409b041664e6ee81beb" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44849 } }
I20260812 00:33:55.972765 10567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:55.972872 10567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81550dfc70ad4409b041664e6ee81beb, State: Initialized, Role: FOLLOWER
I20260812 00:33:55.973538 10567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81550dfc70ad4409b041664e6ee81beb" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44849 } }
I20260812 00:33:55.973816 10567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:33:55.973958 10567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:33:55.974140 10567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:55.976563 10567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81550dfc70ad4409b041664e6ee81beb" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44849 } }
I20260812 00:33:55.977272 10567 leader_election.cc:302] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81550dfc70ad4409b041664e6ee81beb; no voters:
I20260812 00:33:55.977965 10567 leader_election.cc:288] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:33:55.978106 10577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:33:55.978814 10577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [term 1 LEADER]: Becoming Leader. State: Replica: 81550dfc70ad4409b041664e6ee81beb, State: Running, Role: LEADER
I20260812 00:33:55.979394 10577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81550dfc70ad4409b041664e6ee81beb" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44849 } }
I20260812 00:33:55.980175 10567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:33:55.982559 10582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81550dfc70ad4409b041664e6ee81beb. Latest consensus state: current_term: 1 leader_uuid: "81550dfc70ad4409b041664e6ee81beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81550dfc70ad4409b041664e6ee81beb" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44849 } } }
I20260812 00:33:55.983067 10582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [sys.catalog]: This master's current role is: LEADER
I20260812 00:33:55.982534 10579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81550dfc70ad4409b041664e6ee81beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81550dfc70ad4409b041664e6ee81beb" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44849 } } }
I20260812 00:33:55.983632 10579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb [sys.catalog]: This master's current role is: LEADER
I20260812 00:33:55.984341 10590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:33:55.991927 10590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:33:56.007512 10590 catalog_manager.cc:1347] Generated new cluster ID: 948e7125955c423d86ab1e6e984db762
I20260812 00:33:56.007655 10590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:33:56.031817 10590 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:33:56.034000 10590 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:33:56.052946 10590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb: Generated new TSK 0
I20260812 00:33:56.054147 10590 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:33:56.113284 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44849
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:56.469345 10634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:56.469676 10634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:56.469820 10634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:56.477658 10634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:56.477787 10634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:56.477912 10634 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:33:56.478019 10634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:33:56.488823 10634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:44849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:56.490280 10634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:56.491752 10634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:56.506218 10647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:56.506341 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:56.506198 10655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:56.507366 10634 server_base.cc:1034] running on GCE node
I20260812 00:33:56.508052 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:56.509071 10634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:56.510344 10634 hybrid_clock.cc:648] HybridClock initialized: now 1786494836510250 us; error 57 us; skew 500 ppm
I20260812 00:33:56.510659 10634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:56 dist-test-slave-gjw7 krb5kdc[10372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494836, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:56.517477 10634 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:33:56.519069 10634 webserver.cc:460] Webserver started at http://127.7.244.65:40389/ using document root <none> and password file <none>
I20260812 00:33:56.519804 10634 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:56.519905 10634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:56.520217 10634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:56.523347 10634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0"
format_stamp: "Formatted at 2026-08-12 00:33:56 on dist-test-slave-gjw7"
server_key: "f6374ed488e1eb33d26dfcfc79eb25e6"
server_key_iv: "b42feab1f44f59a26c04de55247ffa95"
server_key_version: "encryptionkey@0"
I20260812 00:33:56.524214 10634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0"
format_stamp: "Formatted at 2026-08-12 00:33:56 on dist-test-slave-gjw7"
server_key: "f6374ed488e1eb33d26dfcfc79eb25e6"
server_key_iv: "b42feab1f44f59a26c04de55247ffa95"
server_key_version: "encryptionkey@0"
I20260812 00:33:56.529901 10634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 00:33:56.534029 10671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:56.535804 10634 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:33:56.535990 10634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0"
format_stamp: "Formatted at 2026-08-12 00:33:56 on dist-test-slave-gjw7"
server_key: "f6374ed488e1eb33d26dfcfc79eb25e6"
server_key_iv: "b42feab1f44f59a26c04de55247ffa95"
server_key_version: "encryptionkey@0"
I20260812 00:33:56.536201 10634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:56.563205 10634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:56.567725 10634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:56.567983 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:56.570869 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:56.570964 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:56.571099 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:56.571173 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:33:56 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494836, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:56.597923 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:56.571407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:42915 (local address 127.7.244.126:44849)
0812 00:33:56.572061 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:56.572067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:56.573372 (+ 1305us) server_negotiation.cc:407] Connection header received
0812 00:33:56.575832 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:56.575838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:56.575960 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:56.576087 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:56.579190 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:56.580170 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:56.583453 (+ 3283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:56.584028 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:56.584188 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:56.589920 (+ 5732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:56.589930 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:56.589935 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:56.589991 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:56.594029 (+ 4038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:56.595183 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:56.595191 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:56.595195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:56.595385 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:56.595854 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:56.595860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:56.595863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:56.596274 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:56.596432 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:56.597199 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 00:33:56.597505 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":384,"threads_started":1}
I20260812 00:33:56.599239 10693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:56.572181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44849 (local address 127.7.244.65:42915)
0812 00:33:56.573141 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:56.573198 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:56.575411 (+ 2213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:56.576473 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:56.576493 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:56.577549 (+ 1056us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:56.578887 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:56.578940 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:56.580412 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:56.580419 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:56.583203 (+ 2784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:56.583234 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:56.584348 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:56.584358 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:56.584693 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:56.585719 (+ 1026us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:56.585759 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:56.589508 (+ 3749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:56.594219 (+ 4711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:56.594233 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:56.594255 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:56.594987 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:56.595525 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:56.595529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:56.595531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:56.595748 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:56.596384 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:56.596391 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:56.596804 (+ 413us) client_negotiation.cc:769] Sending connection context
0812 00:33:56.597049 (+ 245us) client_negotiation.cc:240] Negotiation successful
0812 00:33:56.597673 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":419,"threads_started":1}
I20260812 00:33:56.619174 10634 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:41583
I20260812 00:33:56.619189 10917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:41583 every 8 connection(s)
I20260812 00:33:56.620549 10634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:33:56.621106 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:56.606143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:57263 (local address 127.7.244.126:44849)
0812 00:33:56.606374 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:56.606380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:56.606988 (+ 608us) server_negotiation.cc:407] Connection header received
0812 00:33:56.607309 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:56.607316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:56.607393 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:56.607483 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:56.608587 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:56.609610 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:56.611293 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:56.611939 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:56.612091 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:56.614890 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:56.614898 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:56.614903 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:56.614958 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:56.618079 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:56.618963 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:56.618970 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:56.618973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:56.619115 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:56.619578 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:56.619583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:56.619586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:56.619885 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:56.620032 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:56.620490 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 00:33:56.620722 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:33:56.621235 10875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:56.606114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44849 (local address 127.7.244.65:57263)
0812 00:33:56.606853 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:56.606877 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:56.607080 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:56.607547 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:56.607557 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:56.608065 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:56.608384 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:56.608396 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:56.609756 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:56.609761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:56.611102 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:56.611126 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:56.612220 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:56.612231 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:56.612460 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:56.613184 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:56.613209 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:56.614624 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:56.618280 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:56.618286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:56.618290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:56.618827 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:56.619233 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:56.619242 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:56.619246 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:56.619466 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:56.620009 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:56.620014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:56.620210 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 00:33:56.620353 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 00:33:56.620730 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":370,"threads_started":1}
I20260812 00:33:56.623730 10921 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44849
I20260812 00:33:56.624102 10921 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:56.625155 10921 heartbeater.cc:511] Master 127.7.244.126:44849 requested a full tablet report, sending...
I20260812 00:33:56.626274 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 10634
I20260812 00:33:56.626533 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:33:56.627168 8145 external_mini_cluster.cc:1428] Setting key dc1d64fea2cbc119f847d6d653c10fcc
I20260812 00:33:56.627983 10471 ts_manager.cc:194] Registered new tserver with Master: aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583)
I20260812 00:33:56.629868 10471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:42915
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:33:56.673273 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44849
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:57.018743 10933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:57.019096 10933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:57.019246 10933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:57.027418 10933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:57.027588 10933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:57.027716 10933 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:33:57.027832 10933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:33:57.038344 10933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:44849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:57.039860 10933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:57.041419 10933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:57.055449 10953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:57.055540 10946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:57.055536 10944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:57.056461 10933 server_base.cc:1034] running on GCE node
I20260812 00:33:57.057199 10933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:57.058322 10933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:57.059588 10933 hybrid_clock.cc:648] HybridClock initialized: now 1786494837059525 us; error 44 us; skew 500 ppm
I20260812 00:33:57.059923 10933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:57 dist-test-slave-gjw7 krb5kdc[10372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:57.063267 10933 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:33:57.064777 10933 webserver.cc:460] Webserver started at http://127.7.244.66:41181/ using document root <none> and password file <none>
I20260812 00:33:57.065593 10933 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:57.065711 10933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:57.066051 10933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:57.069180 10933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "56f888bce52442598fed56fa788073d2"
format_stamp: "Formatted at 2026-08-12 00:33:57 on dist-test-slave-gjw7"
server_key: "d7d8a128c77c23380688cbd35b418077"
server_key_iv: "6f3b5e4fbabc5cd527988bf3c638f8a6"
server_key_version: "encryptionkey@0"
I20260812 00:33:57.070147 10933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "56f888bce52442598fed56fa788073d2"
format_stamp: "Formatted at 2026-08-12 00:33:57 on dist-test-slave-gjw7"
server_key: "d7d8a128c77c23380688cbd35b418077"
server_key_iv: "6f3b5e4fbabc5cd527988bf3c638f8a6"
server_key_version: "encryptionkey@0"
I20260812 00:33:57.075858 10933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:33:57.079646 10967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:57.081161 10933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:33:57.081382 10933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "56f888bce52442598fed56fa788073d2"
format_stamp: "Formatted at 2026-08-12 00:33:57 on dist-test-slave-gjw7"
server_key: "d7d8a128c77c23380688cbd35b418077"
server_key_iv: "6f3b5e4fbabc5cd527988bf3c638f8a6"
server_key_version: "encryptionkey@0"
I20260812 00:33:57.081620 10933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:57.109663 10933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:57.114665 10933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:57.114980 10933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:57.118106 10933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:57.118212 10933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:57.118361 10933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:57.118422 10933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:33:57 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:57.144456 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.118931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:46759 (local address 127.7.244.126:44849)
0812 00:33:57.119114 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:57.119118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:57.120881 (+ 1763us) server_negotiation.cc:407] Connection header received
0812 00:33:57.123510 (+ 2629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:57.123518 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:57.123662 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:57.123791 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:57.125934 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.127143 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.130439 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.131048 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.131208 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.136561 (+ 5353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:57.136567 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:57.136570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:57.136613 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:57.140205 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.141481 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.141489 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.141493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.141695 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.142253 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.142259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.142262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.142701 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:57.142885 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:57.143754 (+ 869us) server_negotiation.cc:299] Negotiation successful
0812 00:33:57.144050 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 00:33:57.145857 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.119594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44849 (local address 127.7.244.66:46759)
0812 00:33:57.120621 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:57.120691 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:57.123015 (+ 2324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:57.124161 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:57.124172 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:57.124834 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:57.125688 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.125716 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.127376 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.127384 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.130099 (+ 2715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.130133 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.131405 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.131416 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.131825 (+ 409us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.132793 (+ 968us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:57.132824 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:57.136254 (+ 3430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:57.140397 (+ 4143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:57.140411 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:57.140432 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:57.141248 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:57.141862 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:57.141866 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:57.141868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:57.142129 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:57.142821 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:57.142827 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:57.143295 (+ 468us) client_negotiation.cc:769] Sending connection context
0812 00:33:57.143574 (+ 279us) client_negotiation.cc:240] Negotiation successful
0812 00:33:57.144031 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":409,"threads_started":1}
I20260812 00:33:57.167439 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.151761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44849 (local address 127.7.244.66:56477)
0812 00:33:57.152714 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:57.152738 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:57.152889 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:57.153318 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:57.153322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:57.154047 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:57.154602 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.154619 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.156012 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.156021 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.157688 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.157710 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.158546 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.158552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.158703 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.159341 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:57.159359 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:57.160166 (+ 807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:57.164099 (+ 3933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:57.164109 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:57.164114 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:57.164769 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:57.165217 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:57.165223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:57.165227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:57.165441 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:57.166074 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:57.166083 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:57.166359 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 00:33:57.166542 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 00:33:57.166961 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":512,"threads_started":1}
I20260812 00:33:57.167774 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.151779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:56477 (local address 127.7.244.126:44849)
0812 00:33:57.151989 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:57.151996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:57.152859 (+ 863us) server_negotiation.cc:407] Connection header received
0812 00:33:57.153019 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:57.153026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:57.153130 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:57.153258 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:57.154826 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.155840 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.157896 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.158318 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.158446 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.160405 (+ 1959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:57.160411 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:57.160415 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:57.160451 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:57.163851 (+ 3400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.164933 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.164938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.164940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.165095 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.165561 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.165565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.165567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.165947 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:57.166093 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:57.167033 (+ 940us) server_negotiation.cc:299] Negotiation successful
0812 00:33:57.167390 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:33:57.175408 10933 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:39957
I20260812 00:33:57.175439 11201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:39957 every 8 connection(s)
I20260812 00:33:57.176741 10933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:33:57.179209 11202 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44849
I20260812 00:33:57.179471 11202 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:57.180231 11202 heartbeater.cc:511] Master 127.7.244.126:44849 requested a full tablet report, sending...
I20260812 00:33:57.181828 10471 ts_manager.cc:194] Registered new tserver with Master: 56f888bce52442598fed56fa788073d2 (127.7.244.66:39957)
I20260812 00:33:57.182669 10471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:46759
I20260812 00:33:57.185387 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 10933
I20260812 00:33:57.185618 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:33:57.186110 8145 external_mini_cluster.cc:1428] Setting key fdf28b02ed5609122ca2e1f9716baa5d
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:33:57.234978 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44849
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:33835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:33:57.588946 11209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:33:57.589316 11209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:33:57.589471 11209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:33:57.597121 11209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:33:57.597277 11209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:33:57.597435 11209 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:33:57.597551 11209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:33:57.607913 11209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:44849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:33:57.609361 11209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:33:57.610855 11209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:33:57.624553 11216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:57.624526 11219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:33:57.624830 11222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:33:57.626036 11209 server_base.cc:1034] running on GCE node
I20260812 00:33:57.626791 11209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:33:57.627895 11209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:33:57.629132 11209 hybrid_clock.cc:648] HybridClock initialized: now 1786494837629062 us; error 52 us; skew 500 ppm
I20260812 00:33:57.629511 11209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:33:57 dist-test-slave-gjw7 krb5kdc[10372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:33:57.632885 10921 heartbeater.cc:503] Master 127.7.244.126:44849 was elected leader, sending a full tablet report...
I20260812 00:33:57.632897 11209 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:33:57.634902 11209 webserver.cc:460] Webserver started at http://127.7.244.67:45191/ using document root <none> and password file <none>
I20260812 00:33:57.635708 11209 fs_manager.cc:359] Metadata directory not provided
I20260812 00:33:57.635826 11209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:33:57.636265 11209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:33:57.639526 11209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "97d30b7b13d44aa8bd90ef616138a026"
format_stamp: "Formatted at 2026-08-12 00:33:57 on dist-test-slave-gjw7"
server_key: "1e903256649a5e3201974d8dc72ff45a"
server_key_iv: "c317af1ec06a3d4ac68e2c1abbf633c3"
server_key_version: "encryptionkey@0"
I20260812 00:33:57.640479 11209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "97d30b7b13d44aa8bd90ef616138a026"
format_stamp: "Formatted at 2026-08-12 00:33:57 on dist-test-slave-gjw7"
server_key: "1e903256649a5e3201974d8dc72ff45a"
server_key_iv: "c317af1ec06a3d4ac68e2c1abbf633c3"
server_key_version: "encryptionkey@0"
I20260812 00:33:57.646601 11209 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 00:33:57.650758 11238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:57.652379 11209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:33:57.652611 11209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "97d30b7b13d44aa8bd90ef616138a026"
format_stamp: "Formatted at 2026-08-12 00:33:57 on dist-test-slave-gjw7"
server_key: "1e903256649a5e3201974d8dc72ff45a"
server_key_iv: "c317af1ec06a3d4ac68e2c1abbf633c3"
server_key_version: "encryptionkey@0"
I20260812 00:33:57.652873 11209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:33:57.668875 11209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:33:57.673285 11209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:33:57.673655 11209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:33:57.676738 11209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:33:57.676862 11209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:33:57.677047 11209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:33:57.677194 11209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 12 00:33:57 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:57.701198 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.677235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:43189 (local address 127.7.244.126:44849)
0812 00:33:57.677757 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:57.677764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:57.679138 (+ 1374us) server_negotiation.cc:407] Connection header received
0812 00:33:57.681364 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:57.681370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:57.681469 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:57.681603 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:57.683959 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.684959 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.687280 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.687984 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.688148 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.693609 (+ 5461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:57.693618 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:57.693623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:57.693675 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:57.696696 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.698215 (+ 1519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.698222 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.698224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.698387 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.699020 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.699024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.699026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.699316 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:57.699466 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:57.700521 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0812 00:33:57.700763 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":283,"threads_started":1}
I20260812 00:33:57.703287 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.678071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44849 (local address 127.7.244.67:43189)
0812 00:33:57.678954 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:57.678996 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:57.681012 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:57.681876 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:57.681886 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:57.682714 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:57.683735 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.683761 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.685186 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.685190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.687092 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.687110 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.688283 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.688293 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.688607 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.689521 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:57.689547 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:57.693226 (+ 3679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:57.696940 (+ 3714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:57.696954 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:57.696981 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:57.697999 (+ 1018us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:57.698533 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:57.698539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:57.698543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:57.698890 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:57.699438 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:57.699448 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:57.700057 (+ 609us) client_negotiation.cc:769] Sending connection context
0812 00:33:57.700412 (+ 355us) client_negotiation.cc:240] Negotiation successful
0812 00:33:57.701110 (+ 698us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":364,"threads_started":1}
I20260812 00:33:57.724085 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.709688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:60875 (local address 127.7.244.126:44849)
0812 00:33:57.709892 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:57.709898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:57.710645 (+ 747us) server_negotiation.cc:407] Connection header received
0812 00:33:57.710820 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:57.710825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:57.710897 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:57.710985 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:57.712027 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.712775 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.714438 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.715115 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.715265 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.717945 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:57.717950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:57.717953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:57.717991 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:57.721207 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.722203 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.722207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.722210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.722356 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.722777 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.722780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.722782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.723057 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:57.723187 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:57.723635 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 00:33:57.723825 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:33:57.724499 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.709855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44849 (local address 127.7.244.67:60875)
0812 00:33:57.710493 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:57.710514 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:57.710691 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:57.711036 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:57.711041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:57.711549 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:57.711854 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.711865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.712931 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.712935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.714185 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.714200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.715346 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.715356 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.715594 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.716336 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:57.716362 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:57.717675 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:57.721389 (+ 3714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:57.721396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:57.721401 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:57.722058 (+ 657us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:57.722492 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:57.722498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:57.722502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:57.722682 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:57.723158 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:57.723163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:57.723400 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 00:33:57.723564 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 00:33:57.723996 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":350,"threads_started":1}
I20260812 00:33:57.734113 11209 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:45503
I20260812 00:33:57.734148 11475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:45503 every 8 connection(s)
I20260812 00:33:57.735428 11209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:33:57.737566 11476 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44849
I20260812 00:33:57.737800 11476 heartbeater.cc:464] Registering TS with master...
I20260812 00:33:57.738010 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 11209
I20260812 00:33:57.738200 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:33:57.738536 11476 heartbeater.cc:511] Master 127.7.244.126:44849 requested a full tablet report, sending...
I20260812 00:33:57.738607 8145 external_mini_cluster.cc:1428] Setting key 34ba187c4eb074182bbd67a7ed05de70
I20260812 00:33:57.740012 10466 ts_manager.cc:194] Registered new tserver with Master: 97d30b7b13d44aa8bd90ef616138a026 (127.7.244.67:45503)
I20260812 00:33:57.741142 10466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:43189
I20260812 00:33:57.741860 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:33:57 dist-test-slave-gjw7 krb5kdc[10372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:33:57 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:57.793761 11256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.778644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.7.244.126:44849)
0812 00:33:57.778886 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:57.778893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:57.779162 (+ 269us) server_negotiation.cc:407] Connection header received
0812 00:33:57.779367 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:57.779373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:57.779468 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:57.779565 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:57.781197 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.782024 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.783386 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.783821 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.783954 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.788233 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:57.788238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:57.788241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:57.788286 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:57.790830 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.791583 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.791588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.791590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.791763 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:57.792171 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:57.792175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:57.792178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:57.792499 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:57.792796 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:57.793095 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 00:33:57.793399 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 00:33:57.798856 10466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:33:57.806762 10466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:33:57.846422 11501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.821683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35662 (local address 127.7.244.66:39957)
0812 00:33:57.838080 (+ 16397us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:57.838089 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:57.838929 (+ 840us) server_negotiation.cc:407] Connection header received
0812 00:33:57.839051 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:57.839059 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:57.839347 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:57.840175 (+ 828us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:57.840391 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.841682 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.844057 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.844999 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.845142 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.845413 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:57.845905 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 00:33:57.846106 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16231,"thread_start_us":16102,"threads_started":1}
I20260812 00:33:57.846448 11499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.821420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:39957 (local address 127.0.0.1:35662)
0812 00:33:57.838546 (+ 17126us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:57.838566 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:57.838688 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:57.839609 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:57.839619 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:57.839652 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:57.840179 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.840197 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.841818 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.841824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.843857 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.843874 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.845248 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.845256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.845591 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.845655 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 00:33:57.845817 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 00:33:57.846027 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16971,"mutex_wait_us":264,"thread_start_us":16607,"threads_started":1}
I20260812 00:33:57.848065 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.821614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.7.244.67:45503)
0812 00:33:57.837615 (+ 16001us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:57.837625 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:57.839099 (+ 1474us) server_negotiation.cc:407] Connection header received
0812 00:33:57.839204 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:57.839210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:57.839445 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:57.839678 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:57.841135 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.842770 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.845499 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.846985 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.847101 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.847283 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:57.847597 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 00:33:57.847738 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15781,"thread_start_us":16487,"threads_started":1}
I20260812 00:33:57.848129 11502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.821406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:45503 (local address 127.0.0.1:57408)
0812 00:33:57.838310 (+ 16904us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:57.838377 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:57.838689 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:57.839999 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:57.840004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:57.840028 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:57.840899 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.840913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.842935 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.842939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.845205 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.845227 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.847192 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.847199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.847399 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.847447 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 00:33:57.847546 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:33:57.847690 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16566,"mutex_wait_us":25,"thread_start_us":18780,"threads_started":1}
I20260812 00:33:57.849505 11497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.822243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:41583 (local address 127.0.0.1:42214)
0812 00:33:57.838769 (+ 16526us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:57.838788 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:57.839042 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:57.839975 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:57.839979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:57.840005 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:57.840597 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.840610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.842834 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.842840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.845534 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:57.845547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.847909 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.847916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:57.848117 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.848859 (+ 742us) client_negotiation.cc:769] Sending connection context
0812 00:33:57.848942 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 00:33:57.849075 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16392,"mutex_wait_us":1,"spinlock_wait_cycles":7040,"thread_start_us":16612,"threads_started":1}
I20260812 00:33:57.849936 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:57.822954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42214 (local address 127.7.244.65:41583)
0812 00:33:57.831546 (+ 8592us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:57.831558 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:57.838925 (+ 7367us) server_negotiation.cc:407] Connection header received
0812 00:33:57.839265 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:57.839274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:57.839500 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:57.839681 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:57.840860 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.842434 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.845713 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:57.847680 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:57.847817 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:57.848045 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:57.849435 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0812 00:33:57.849627 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8355,"thread_start_us":8183,"threads_started":1}
I20260812 00:33:57.852602 11334 tablet_service.cc:1467] Processing CreateTablet for tablet 616bf60f33be40f6a63497cf0fea05c2 (DEFAULT_TABLE table=test-table [id=b8f6d8308d46433aad6f0f73d32cef65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:57.852607 11071 tablet_service.cc:1467] Processing CreateTablet for tablet 616bf60f33be40f6a63497cf0fea05c2 (DEFAULT_TABLE table=test-table [id=b8f6d8308d46433aad6f0f73d32cef65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:57.853545 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 616bf60f33be40f6a63497cf0fea05c2 (DEFAULT_TABLE table=test-table [id=b8f6d8308d46433aad6f0f73d32cef65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:57.854290 11071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616bf60f33be40f6a63497cf0fea05c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:57.854590 11334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616bf60f33be40f6a63497cf0fea05c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:57.855026 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 616bf60f33be40f6a63497cf0fea05c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:57.867863 11504 tablet_bootstrap.cc:492] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Bootstrap starting.
I20260812 00:33:57.871421 11504 tablet_bootstrap.cc:654] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:57.872965 11504 log.cc:824] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Log is configured to *not* fsync() on all Append() calls
I20260812 00:33:57.876433 11504 tablet_bootstrap.cc:492] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: No bootstrap required, opened a new log
I20260812 00:33:57.876740 11504 ts_tablet_manager.cc:1397] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 00:33:57.880802 11505 tablet_bootstrap.cc:492] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Bootstrap starting.
I20260812 00:33:57.881179 11507 tablet_bootstrap.cc:492] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Bootstrap starting.
I20260812 00:33:57.885532 11507 tablet_bootstrap.cc:654] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:57.886969 11505 tablet_bootstrap.cc:654] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:57.887243 11507 log.cc:824] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:33:57.888448 11505 log.cc:824] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Log is configured to *not* fsync() on all Append() calls
I20260812 00:33:57.892241 11505 tablet_bootstrap.cc:492] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: No bootstrap required, opened a new log
I20260812 00:33:57.893604 11505 ts_tablet_manager.cc:1397] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.008s
I20260812 00:33:57.894733 11507 tablet_bootstrap.cc:492] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: No bootstrap required, opened a new log
I20260812 00:33:57.895066 11507 ts_tablet_manager.cc:1397] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.000s
I20260812 00:33:57.895251 11504 raft_consensus.cc:348] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:57.895773 11504 raft_consensus.cc:374] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:57.895881 11504 raft_consensus.cc:729] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d30b7b13d44aa8bd90ef616138a026, State: Initialized, Role: FOLLOWER
I20260812 00:33:57.896689 11504 consensus_queue.cc:260] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:57.898042 11476 heartbeater.cc:503] Master 127.7.244.126:44849 was elected leader, sending a full tablet report...
I20260812 00:33:57.898746 11504 ts_tablet_manager.cc:1428] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Time spent starting tablet: real 0.022s user 0.014s sys 0.001s
I20260812 00:33:57.903046 11505 raft_consensus.cc:348] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:57.903651 11505 raft_consensus.cc:374] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:57.903775 11505 raft_consensus.cc:729] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f888bce52442598fed56fa788073d2, State: Initialized, Role: FOLLOWER
I20260812 00:33:57.904469 11505 consensus_queue.cc:260] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:57.905810 11202 heartbeater.cc:503] Master 127.7.244.126:44849 was elected leader, sending a full tablet report...
I20260812 00:33:57.905802 11507 raft_consensus.cc:348] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:57.906365 11507 raft_consensus.cc:374] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:57.906491 11507 raft_consensus.cc:729] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9ac64c093141a5b099fdcfd1ec4dd0, State: Initialized, Role: FOLLOWER
I20260812 00:33:57.906769 11505 ts_tablet_manager.cc:1428] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260812 00:33:57.907317 11507 consensus_queue.cc:260] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:57.909556 11507 ts_tablet_manager.cc:1428] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
W20260812 00:33:57.928941 11204 tablet.cc:2369] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:33:57.987084 11477 tablet.cc:2369] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:33:58.101398 11512 raft_consensus.cc:482] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:33:58.101809 11512 raft_consensus.cc:504] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:58.103713 11512 leader_election.cc:288] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97d30b7b13d44aa8bd90ef616138a026 (127.7.244.67:45503), aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583)
I20260812 00:33:58.106945 11511 raft_consensus.cc:482] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:33:58.107275 11511 raft_consensus.cc:504] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:58.109563 11511 leader_election.cc:288] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56f888bce52442598fed56fa788073d2 (127.7.244.66:39957), aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583)
I20260812 00:33:58.112351 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.104370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:46035 (local address 127.7.244.67:45503)
0812 00:33:58.104604 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.104611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.105357 (+ 746us) server_negotiation.cc:407] Connection header received
0812 00:33:58.105547 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.105551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.105632 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.105718 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:58.106395 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.107493 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.109772 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.111066 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.111209 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.111288 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.111913 (+ 625us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.112064 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 00:33:58.112932 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.104108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:36873 (local address 127.7.244.65:41583)
0812 00:33:58.104318 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.104323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.105013 (+ 690us) server_negotiation.cc:407] Connection header received
0812 00:33:58.105356 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.105362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.105442 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.105530 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:58.106246 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.107429 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.110059 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.111404 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.111571 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.111659 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.112330 (+ 671us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.112638 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 00:33:58.113183 11524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.104128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:45503 (local address 127.7.244.66:46035)
0812 00:33:58.105230 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:58.105245 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:58.105416 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:58.105750 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:58.105754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:58.105777 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:58.106244 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.106257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.107668 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.107674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.109564 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.109586 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.111501 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.111508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.111678 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.111712 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 00:33:58.112567 (+ 855us) client_negotiation.cc:240] Negotiation successful
0812 00:33:58.112729 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":962,"mutex_wait_us":111,"thread_start_us":761,"threads_started":1}
I20260812 00:33:58.114014 11522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.104041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:41583 (local address 127.7.244.66:36873)
0812 00:33:58.104892 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:58.104908 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:58.105083 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:58.105571 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:58.105576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:58.105601 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:58.106062 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.106077 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.107595 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.107601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.109840 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.109860 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.111892 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.111900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.112115 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.112152 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 00:33:58.113349 (+ 1197us) client_negotiation.cc:240] Negotiation successful
0812 00:33:58.113559 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":24,"thread_start_us":484,"threads_started":1}
I20260812 00:33:58.114360 11387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616bf60f33be40f6a63497cf0fea05c2" candidate_uuid: "56f888bce52442598fed56fa788073d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d30b7b13d44aa8bd90ef616138a026" is_pre_election: true
I20260812 00:33:58.114866 11387 raft_consensus.cc:2454] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56f888bce52442598fed56fa788073d2 in term 0.
I20260812 00:33:58.115717 10968 leader_election.cc:302] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f888bce52442598fed56fa788073d2, 97d30b7b13d44aa8bd90ef616138a026; no voters:
I20260812 00:33:58.115856 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616bf60f33be40f6a63497cf0fea05c2" candidate_uuid: "56f888bce52442598fed56fa788073d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" is_pre_election: true
I20260812 00:33:58.116253 11512 raft_consensus.cc:2781] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:33:58.116398 11512 raft_consensus.cc:482] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:33:58.116518 11512 raft_consensus.cc:3037] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.116539 10833 raft_consensus.cc:2454] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56f888bce52442598fed56fa788073d2 in term 0.
I20260812 00:33:58.119459 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.109934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:39957 (local address 127.7.244.67:38627)
0812 00:33:58.110413 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:58.110438 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:58.110562 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:58.112252 (+ 1690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:58.112256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:58.112289 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:58.112517 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.112529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.114112 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.114117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.116325 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.116347 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.118416 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.118424 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.118684 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.118725 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 00:33:58.118840 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 00:33:58.119034 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325}
I20260812 00:33:58.120477 11501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.110911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:38627 (local address 127.7.244.66:39957)
0812 00:33:58.111102 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.111107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.111125 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:33:58.111188 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.111193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.111265 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.111358 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:58.112882 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.113970 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.116581 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.118021 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.118209 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.118299 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.119981 (+ 1682us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.120159 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:33:58.120333 11512 raft_consensus.cc:504] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:58.121622 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616bf60f33be40f6a63497cf0fea05c2" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56f888bce52442598fed56fa788073d2" is_pre_election: true
I20260812 00:33:58.122203 11121 raft_consensus.cc:2379] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97d30b7b13d44aa8bd90ef616138a026 in current term 1: Already voted for candidate 56f888bce52442598fed56fa788073d2 in this term.
I20260812 00:33:58.122349 11387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616bf60f33be40f6a63497cf0fea05c2" candidate_uuid: "56f888bce52442598fed56fa788073d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97d30b7b13d44aa8bd90ef616138a026"
I20260812 00:33:58.122633 11387 raft_consensus.cc:3037] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.126775 11387 raft_consensus.cc:2454] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56f888bce52442598fed56fa788073d2 in term 1.
I20260812 00:33:58.127389 10968 leader_election.cc:302] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f888bce52442598fed56fa788073d2, 97d30b7b13d44aa8bd90ef616138a026; no voters:
I20260812 00:33:58.128228 11530 raft_consensus.cc:2781] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:33:58.128607 11503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.111821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:46127 (local address 127.7.244.65:41583)
0812 00:33:58.113284 (+ 1463us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.113307 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.120944 (+ 7637us) server_negotiation.cc:407] Connection header received
0812 00:33:58.121028 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.121033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.121170 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.121260 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:33:58.121963 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.123055 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.125850 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.127378 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.127585 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.127671 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.128084 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.128249 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1345,"threads_started":1}
I20260812 00:33:58.128737 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616bf60f33be40f6a63497cf0fea05c2" candidate_uuid: "56f888bce52442598fed56fa788073d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0"
I20260812 00:33:58.128937 10833 raft_consensus.cc:3037] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.129161 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.110095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:41583 (local address 127.7.244.67:46127)
0812 00:33:58.119759 (+ 9664us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:58.119781 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:58.120812 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:58.121482 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:58.121487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:58.121518 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:33:58.121718 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.121732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.123247 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.123254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.125516 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.125541 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.127609 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.127616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.127844 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.127882 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 00:33:58.128617 (+ 735us) client_negotiation.cc:240] Negotiation successful
0812 00:33:58.128804 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9549,"thread_start_us":16340,"threads_started":1}
I20260812 00:33:58.130129 11512 leader_election.cc:288] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [CANDIDATE]: Term 1 election: Requested vote from peers 97d30b7b13d44aa8bd90ef616138a026 (127.7.244.67:45503), aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583)
I20260812 00:33:58.130070 10832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "616bf60f33be40f6a63497cf0fea05c2" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" is_pre_election: true
I20260812 00:33:58.130687 11530 raft_consensus.cc:686] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 1 LEADER]: Becoming Leader. State: Replica: 56f888bce52442598fed56fa788073d2, State: Running, Role: LEADER
I20260812 00:33:58.131392 11244 leader_election.cc:302] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97d30b7b13d44aa8bd90ef616138a026; no voters: 56f888bce52442598fed56fa788073d2, aa9ac64c093141a5b099fdcfd1ec4dd0
I20260812 00:33:58.131441 11530 consensus_queue.cc:237] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } }
I20260812 00:33:58.132043 11511 raft_consensus.cc:2726] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 00:33:58.132316 10833 raft_consensus.cc:2454] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56f888bce52442598fed56fa788073d2 in term 1.
W20260812 00:33:58.133167 10924 tablet.cc:2369] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:33:58.136826 10466 catalog_manager.cc:5495] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56f888bce52442598fed56fa788073d2 (127.7.244.66). New cstate: current_term: 1 leader_uuid: "56f888bce52442598fed56fa788073d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } health_report { overall_health: UNKNOWN } } }
I20260812 00:33:58.161019 11501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.154870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.7.244.66:39957)
0812 00:33:58.155086 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.155092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.155197 (+ 105us) server_negotiation.cc:407] Connection header received
0812 00:33:58.155566 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.155570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.155656 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.155788 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:33:58.156220 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.157033 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.158529 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.158994 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.159115 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.159706 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:33:58.160307 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:33:58.160448 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.160612 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.160708 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:33:58.169975 10833 raft_consensus.cc:1261] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Refusing update from remote peer 56f888bce52442598fed56fa788073d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:33:58.170015 11387 raft_consensus.cc:1261] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Refusing update from remote peer 56f888bce52442598fed56fa788073d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:33:58.170977 11530 consensus_queue.cc:1035] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:33:58.171324 11532 consensus_queue.cc:1035] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:33:58.190332 11540 mvcc.cc:205] Tried to move back new op lower bound from 7317482857133547520 to 7317482856991817728. Current Snapshot: MvccSnapshot[applied={T|T < 7317482857133547520}]
I20260812 00:33:58.208444 11543 mvcc.cc:205] Tried to move back new op lower bound from 7317482857133547520 to 7317482856991817728. Current Snapshot: MvccSnapshot[applied={T|T < 7317482857133547520}]
I20260812 00:33:58.220604 10471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:33:58.220919 10471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:33:58.224905 10471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb: Sending DeleteTablet for 3 replicas of tablet 616bf60f33be40f6a63497cf0fea05c2
I20260812 00:33:58.226281 11334 tablet_service.cc:1514] Processing DeleteTablet for tablet 616bf60f33be40f6a63497cf0fea05c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:33:58 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:57408
I20260812 00:33:58.226418 11071 tablet_service.cc:1514] Processing DeleteTablet for tablet 616bf60f33be40f6a63497cf0fea05c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:33:58 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:35662
I20260812 00:33:58.226595 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet 616bf60f33be40f6a63497cf0fea05c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:33:58 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:42214
I20260812 00:33:58.232456 10471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:33:58.234274 10471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:33:58.238940 11557 tablet_replica.cc:330] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: stopping tablet replica
I20260812 00:33:58.239645 11557 raft_consensus.cc:2229] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:33:58.245811 11556 tablet_replica.cc:330] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: stopping tablet replica
I20260812 00:33:58.245913 11333 tablet_service.cc:1467] Processing CreateTablet for tablet abaa61c108e3410b86f5a9891d54e510 (DEFAULT_TABLE table=test-table [id=2636887fe0e345b296ea36a0d99a1e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:58.246454 11333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abaa61c108e3410b86f5a9891d54e510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:58.246836 11071 tablet_service.cc:1467] Processing CreateTablet for tablet abaa61c108e3410b86f5a9891d54e510 (DEFAULT_TABLE table=test-table [id=2636887fe0e345b296ea36a0d99a1e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:58.246928 10774 tablet_service.cc:1467] Processing CreateTablet for tablet abaa61c108e3410b86f5a9891d54e510 (DEFAULT_TABLE table=test-table [id=2636887fe0e345b296ea36a0d99a1e02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:33:58.247511 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abaa61c108e3410b86f5a9891d54e510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:58.247530 11071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abaa61c108e3410b86f5a9891d54e510. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:58.253436 11504 tablet_bootstrap.cc:492] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: Bootstrap starting.
I20260812 00:33:58.254135 11551 tablet_replica.cc:330] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: stopping tablet replica
I20260812 00:33:58.254513 11551 raft_consensus.cc:2229] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:33:58.254908 11551 raft_consensus.cc:2258] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:33:58.255187 11557 raft_consensus.cc:2258] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:33:58.255349 11504 tablet_bootstrap.cc:654] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:58.257259 11507 tablet_bootstrap.cc:492] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: Bootstrap starting.
I20260812 00:33:58.258114 11557 ts_tablet_manager.cc:1905] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:33:58.259239 11507 tablet_bootstrap.cc:654] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:58.261659 11507 tablet_bootstrap.cc:492] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: No bootstrap required, opened a new log
I20260812 00:33:58.261822 11507 ts_tablet_manager.cc:1397] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 00:33:58.262719 11507 raft_consensus.cc:348] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.262959 11507 raft_consensus.cc:374] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:58.263060 11507 raft_consensus.cc:729] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9ac64c093141a5b099fdcfd1ec4dd0, State: Initialized, Role: FOLLOWER
I20260812 00:33:58.263163 11551 ts_tablet_manager.cc:1905] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:33:58.263342 11507 consensus_queue.cc:260] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.263101 11556 raft_consensus.cc:2229] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:33:58.264101 11507 ts_tablet_manager.cc:1428] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 00:33:58.264410 11556 raft_consensus.cc:2258] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:33:58.268797 11556 ts_tablet_manager.cc:1905] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:33:58.270290 11551 ts_tablet_manager.cc:1918] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:33:58.270455 11551 log.cc:1196] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/616bf60f33be40f6a63497cf0fea05c2
I20260812 00:33:58.270916 11551 ts_tablet_manager.cc:1939] T 616bf60f33be40f6a63497cf0fea05c2 P 97d30b7b13d44aa8bd90ef616138a026: Deleting consensus metadata
I20260812 00:33:58.272545 10430 catalog_manager.cc:4841] TS 97d30b7b13d44aa8bd90ef616138a026 (127.7.244.67:45503): tablet 616bf60f33be40f6a63497cf0fea05c2 (table test-table [id=b8f6d8308d46433aad6f0f73d32cef65]) successfully deleted
I20260812 00:33:58.275355 11557 ts_tablet_manager.cc:1918] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:33:58.275550 11557 log.cc:1196] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/616bf60f33be40f6a63497cf0fea05c2
I20260812 00:33:58.276178 11557 ts_tablet_manager.cc:1939] T 616bf60f33be40f6a63497cf0fea05c2 P aa9ac64c093141a5b099fdcfd1ec4dd0: Deleting consensus metadata
I20260812 00:33:58.277793 11504 tablet_bootstrap.cc:492] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: No bootstrap required, opened a new log
I20260812 00:33:58.277974 11504 ts_tablet_manager.cc:1397] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.001s
I20260812 00:33:58.278235 10434 catalog_manager.cc:4841] TS aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583): tablet 616bf60f33be40f6a63497cf0fea05c2 (table test-table [id=b8f6d8308d46433aad6f0f73d32cef65]) successfully deleted
I20260812 00:33:58.278863 11504 raft_consensus.cc:348] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.279078 11505 tablet_bootstrap.cc:492] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: Bootstrap starting.
I20260812 00:33:58.279121 11504 raft_consensus.cc:374] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:58.279218 11504 raft_consensus.cc:729] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d30b7b13d44aa8bd90ef616138a026, State: Initialized, Role: FOLLOWER
I20260812 00:33:58.279546 11504 consensus_queue.cc:260] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.281020 11505 tablet_bootstrap.cc:654] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:58.282858 11504 ts_tablet_manager.cc:1428] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260812 00:33:58.283578 11556 ts_tablet_manager.cc:1918] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:33:58.283905 11556 log.cc:1196] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/616bf60f33be40f6a63497cf0fea05c2
I20260812 00:33:58.284581 11556 ts_tablet_manager.cc:1939] T 616bf60f33be40f6a63497cf0fea05c2 P 56f888bce52442598fed56fa788073d2: Deleting consensus metadata
I20260812 00:33:58.287094 10437 catalog_manager.cc:4841] TS 56f888bce52442598fed56fa788073d2 (127.7.244.66:39957): tablet 616bf60f33be40f6a63497cf0fea05c2 (table test-table [id=b8f6d8308d46433aad6f0f73d32cef65]) successfully deleted
I20260812 00:33:58.288004 11505 tablet_bootstrap.cc:492] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: No bootstrap required, opened a new log
I20260812 00:33:58.288172 11505 ts_tablet_manager.cc:1397] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260812 00:33:58.289063 11505 raft_consensus.cc:348] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.289347 11505 raft_consensus.cc:374] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:58.289471 11505 raft_consensus.cc:729] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f888bce52442598fed56fa788073d2, State: Initialized, Role: FOLLOWER
I20260812 00:33:58.289744 11505 consensus_queue.cc:260] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.290378 11505 ts_tablet_manager.cc:1428] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:33:58.418695 11511 raft_consensus.cc:482] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:33:58.418949 11511 raft_consensus.cc:504] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.419688 11511 leader_election.cc:288] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583), 56f888bce52442598fed56fa788073d2 (127.7.244.66:39957)
I20260812 00:33:58.420504 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abaa61c108e3410b86f5a9891d54e510" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" is_pre_election: true
I20260812 00:33:58.420563 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abaa61c108e3410b86f5a9891d54e510" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56f888bce52442598fed56fa788073d2" is_pre_election: true
I20260812 00:33:58.420792 10833 raft_consensus.cc:2454] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d30b7b13d44aa8bd90ef616138a026 in term 0.
I20260812 00:33:58.420876 11121 raft_consensus.cc:2454] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d30b7b13d44aa8bd90ef616138a026 in term 0.
I20260812 00:33:58.421265 11244 leader_election.cc:302] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d30b7b13d44aa8bd90ef616138a026, aa9ac64c093141a5b099fdcfd1ec4dd0; no voters:
I20260812 00:33:58.421656 11511 raft_consensus.cc:2781] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:33:58.421772 11511 raft_consensus.cc:482] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:33:58.421859 11511 raft_consensus.cc:3037] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.424054 11511 raft_consensus.cc:504] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.424659 11511 leader_election.cc:288] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 election: Requested vote from peers aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583), 56f888bce52442598fed56fa788073d2 (127.7.244.66:39957)
I20260812 00:33:58.425307 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abaa61c108e3410b86f5a9891d54e510" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56f888bce52442598fed56fa788073d2"
I20260812 00:33:58.425405 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abaa61c108e3410b86f5a9891d54e510" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0"
I20260812 00:33:58.425573 11121 raft_consensus.cc:3037] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.425681 10833 raft_consensus.cc:3037] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.427956 11121 raft_consensus.cc:2454] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d30b7b13d44aa8bd90ef616138a026 in term 1.
I20260812 00:33:58.428494 11246 leader_election.cc:302] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f888bce52442598fed56fa788073d2, 97d30b7b13d44aa8bd90ef616138a026; no voters:
I20260812 00:33:58.428828 11511 raft_consensus.cc:2781] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:33:58.428875 10833 raft_consensus.cc:2454] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d30b7b13d44aa8bd90ef616138a026 in term 1.
I20260812 00:33:58.429317 11511 raft_consensus.cc:686] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 LEADER]: Becoming Leader. State: Replica: 97d30b7b13d44aa8bd90ef616138a026, State: Running, Role: LEADER
I20260812 00:33:58.429744 11511 consensus_queue.cc:237] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.433681 10471 catalog_manager.cc:5495] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97d30b7b13d44aa8bd90ef616138a026 (127.7.244.67). New cstate: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } health_report { overall_health: HEALTHY } } }
Aug 12 00:33:58 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494835, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:33:58.483770 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.468335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44849 (local address 127.0.0.1:39914)
0812 00:33:58.469051 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:58.469076 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:58.469284 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:58.469684 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:58.469689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:58.470413 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:33:58.470624 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.470635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.471757 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.471761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.472720 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.472734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.473549 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.473555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.473729 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.474600 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:33:58.474620 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:33:58.477531 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:33:58.480912 (+ 3381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:58.480931 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:58.480939 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:58.481560 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:58.481930 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:33:58.481936 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:33:58.481940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:33:58.482097 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:33:58.482655 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:33:58.482665 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:33:58.482853 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 00:33:58.483003 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 00:33:58.483310 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":367,"threads_started":1}
I20260812 00:33:58.483804 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.468358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.7.244.126:44849)
0812 00:33:58.469405 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.469410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.469431 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:33:58.469475 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.469479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.469532 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.469619 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:58.470809 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.471610 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.472889 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.473380 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.473498 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.477773 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:58.477779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:58.477782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:58.477820 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:58.480757 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:58.481676 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:58.481681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:58.481685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:58.481818 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:58.482213 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:58.482216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:58.482218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:58.482519 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:58.482657 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.483134 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.483450 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":898,"thread_start_us":738,"threads_started":1}
I20260812 00:33:58.489866 10471 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:39914:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 00:33:58.490908 10471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:33:58.498278 10774 tablet_service.cc:1467] Processing CreateTablet for tablet 8641a84695834f9a9cf2cc74500ac7d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6d7d89df08374c618343ad8078275cfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:33:58.498337 11071 tablet_service.cc:1467] Processing CreateTablet for tablet 8641a84695834f9a9cf2cc74500ac7d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6d7d89df08374c618343ad8078275cfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:33:58.498983 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8641a84695834f9a9cf2cc74500ac7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:58.499018 11071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8641a84695834f9a9cf2cc74500ac7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:58.502457 11334 tablet_service.cc:1467] Processing CreateTablet for tablet 8641a84695834f9a9cf2cc74500ac7d0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6d7d89df08374c618343ad8078275cfb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:33:58.503120 11334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8641a84695834f9a9cf2cc74500ac7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:33:58.511068 11504 tablet_bootstrap.cc:492] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: Bootstrap starting.
I20260812 00:33:58.512977 11504 tablet_bootstrap.cc:654] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:58.515081 11507 tablet_bootstrap.cc:492] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: Bootstrap starting.
I20260812 00:33:58.515307 11505 tablet_bootstrap.cc:492] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: Bootstrap starting.
I20260812 00:33:58.516826 11507 tablet_bootstrap.cc:654] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:58.517343 11505 tablet_bootstrap.cc:654] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: Neither blocks nor log segments found. Creating new log.
I20260812 00:33:58.520107 11504 tablet_bootstrap.cc:492] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: No bootstrap required, opened a new log
I20260812 00:33:58.520263 11507 tablet_bootstrap.cc:492] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: No bootstrap required, opened a new log
I20260812 00:33:58.520331 11504 ts_tablet_manager.cc:1397] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260812 00:33:58.520416 11505 tablet_bootstrap.cc:492] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: No bootstrap required, opened a new log
I20260812 00:33:58.520442 11507 ts_tablet_manager.cc:1397] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 00:33:58.520603 11505 ts_tablet_manager.cc:1397] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260812 00:33:58.521144 11504 raft_consensus.cc:348] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.521137 11507 raft_consensus.cc:348] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.521446 11507 raft_consensus.cc:374] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:58.521446 11504 raft_consensus.cc:374] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:58.521526 11504 raft_consensus.cc:729] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d30b7b13d44aa8bd90ef616138a026, State: Initialized, Role: FOLLOWER
I20260812 00:33:58.521528 11507 raft_consensus.cc:729] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa9ac64c093141a5b099fdcfd1ec4dd0, State: Initialized, Role: FOLLOWER
I20260812 00:33:58.521474 11505 raft_consensus.cc:348] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.521786 11504 consensus_queue.cc:260] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.521951 11505 raft_consensus.cc:374] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:33:58.521863 11507 consensus_queue.cc:260] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.522045 11505 raft_consensus.cc:729] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56f888bce52442598fed56fa788073d2, State: Initialized, Role: FOLLOWER
I20260812 00:33:58.522240 11511 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.522318 11505 consensus_queue.cc:260] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.522274 11517 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.522557 11511 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:33:58.522589 11517 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:33:58.522930 11505 ts_tablet_manager.cc:1428] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 00:33:58.523365 11532 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.523756 11532 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:33:58.524385 11507 ts_tablet_manager.cc:1428] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 00:33:58.524892 11504 ts_tablet_manager.cc:1428] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 00:33:58.651644 11511 raft_consensus.cc:482] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:33:58.651916 11511 raft_consensus.cc:504] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.652796 11511 leader_election.cc:288] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583), 56f888bce52442598fed56fa788073d2 (127.7.244.66:39957)
I20260812 00:33:58.653574 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8641a84695834f9a9cf2cc74500ac7d0" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" is_pre_election: true
I20260812 00:33:58.653641 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8641a84695834f9a9cf2cc74500ac7d0" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56f888bce52442598fed56fa788073d2" is_pre_election: true
I20260812 00:33:58.653879 10833 raft_consensus.cc:2454] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d30b7b13d44aa8bd90ef616138a026 in term 0.
I20260812 00:33:58.653945 11121 raft_consensus.cc:2454] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d30b7b13d44aa8bd90ef616138a026 in term 0.
I20260812 00:33:58.654398 11246 leader_election.cc:302] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f888bce52442598fed56fa788073d2, 97d30b7b13d44aa8bd90ef616138a026; no voters:
I20260812 00:33:58.654779 11511 raft_consensus.cc:2781] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:33:58.654886 11511 raft_consensus.cc:482] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:33:58.654991 11511 raft_consensus.cc:3037] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.658133 11511 raft_consensus.cc:504] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.659278 11511 leader_election.cc:288] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 election: Requested vote from peers aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583), 56f888bce52442598fed56fa788073d2 (127.7.244.66:39957)
I20260812 00:33:58.659808 11121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8641a84695834f9a9cf2cc74500ac7d0" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56f888bce52442598fed56fa788073d2"
I20260812 00:33:58.659790 10833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8641a84695834f9a9cf2cc74500ac7d0" candidate_uuid: "97d30b7b13d44aa8bd90ef616138a026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0"
I20260812 00:33:58.660105 11121 raft_consensus.cc:3037] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.660102 10833 raft_consensus.cc:3037] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:33:58.663022 11121 raft_consensus.cc:2454] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d30b7b13d44aa8bd90ef616138a026 in term 1.
I20260812 00:33:58.663614 11246 leader_election.cc:302] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56f888bce52442598fed56fa788073d2, 97d30b7b13d44aa8bd90ef616138a026; no voters:
I20260812 00:33:58.663962 11511 raft_consensus.cc:2781] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:33:58.664009 10833 raft_consensus.cc:2454] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d30b7b13d44aa8bd90ef616138a026 in term 1.
I20260812 00:33:58.664160 11511 raft_consensus.cc:686] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 LEADER]: Becoming Leader. State: Replica: 97d30b7b13d44aa8bd90ef616138a026, State: Running, Role: LEADER
I20260812 00:33:58.664460 11511 consensus_queue.cc:237] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } }
I20260812 00:33:58.666126 11563 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: TxnStatusTablet state changed. Reason: New leader 97d30b7b13d44aa8bd90ef616138a026. Latest consensus state: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.666424 11563 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:33:58.667547 11591 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 00:33:58.667800 11591 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 00:33:58.668123 10462 catalog_manager.cc:5495] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97d30b7b13d44aa8bd90ef616138a026 (127.7.244.67). New cstate: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } health_report { overall_health: HEALTHY } } }
I20260812 00:33:58.725278 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.718197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:45503 (local address 127.0.0.1:57412)
0812 00:33:58.718509 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:58.718539 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:58.718732 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:58.719415 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:58.719419 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:58.719430 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:33:58.719617 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.719625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.721158 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.721163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.722318 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.722335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.723376 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.723380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.723559 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.723639 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:33:58.724810 (+ 1171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:33:58.724826 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 00:33:58.724891 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 00:33:58.725023 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:33:58.725423 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.718259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.7.244.67:45503)
0812 00:33:58.718913 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.718920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.718941 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:33:58.719054 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.719060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.719171 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.719359 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:33:58.719761 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.720994 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.722482 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.723214 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.723356 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.723840 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:33:58.724694 (+ 854us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:33:58.724839 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.724940 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.725036 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":337,"threads_started":1}
I20260812 00:33:58.730290 10833 raft_consensus.cc:1261] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Refusing update from remote peer 97d30b7b13d44aa8bd90ef616138a026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:33:58.730495 11121 raft_consensus.cc:1261] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Refusing update from remote peer 97d30b7b13d44aa8bd90ef616138a026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:33:58.731074 11511 consensus_queue.cc:1035] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:33:58.731439 11563 consensus_queue.cc:1035] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:33:58.738250 11517 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: TxnStatusTablet state changed. Reason: New leader 97d30b7b13d44aa8bd90ef616138a026. Latest consensus state: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.738559 11517 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:33:58.747946 11610 mvcc.cc:205] Tried to move back new op lower bound from 7317482859430965248 to 7317482859172360192. Current Snapshot: MvccSnapshot[applied={T|T < 7317482859430965248}]
I20260812 00:33:58.750514 11517 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.750828 11517 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P aa9ac64c093141a5b099fdcfd1ec4dd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:33:58.758140 11532 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: TxnStatusTablet state changed. Reason: New leader 97d30b7b13d44aa8bd90ef616138a026. Latest consensus state: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.758484 11532 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:33:58.761713 11563 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.762035 11563 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:33:58.762446 11563 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.762709 11563 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P 97d30b7b13d44aa8bd90ef616138a026: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:33:58.779878 11532 tablet_replica.cc:439] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97d30b7b13d44aa8bd90ef616138a026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 } } peers { permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 } } peers { permanent_uuid: "97d30b7b13d44aa8bd90ef616138a026" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45503 } } }
I20260812 00:33:58.780210 11532 tablet_replica.cc:442] T 8641a84695834f9a9cf2cc74500ac7d0 P 56f888bce52442598fed56fa788073d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:33:58.787412 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.781594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.7.244.67:45503)
0812 00:33:58.781763 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.781769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.781859 (+ 90us) server_negotiation.cc:407] Connection header received
0812 00:33:58.782172 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.782179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.782263 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.782355 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:33:58.782889 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.783680 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.785167 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.785854 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.785983 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.786421 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:33:58.786543 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:33:58.786620 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.786938 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.787052 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 00:33:58.806330 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.799381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:47163 (local address 127.7.244.67:45503)
0812 00:33:58.799567 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.799571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.800224 (+ 653us) server_negotiation.cc:407] Connection header received
0812 00:33:58.800419 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.800425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.800517 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.800628 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:33:58.801227 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.802419 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.804063 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.804756 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.804898 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.805455 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:33:58.805629 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:33:58.805715 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.805865 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.805961 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 00:33:58.806413 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.799454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:45503 (local address 127.7.244.67:47163)
0812 00:33:58.800112 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 00:33:58.800128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:33:58.800282 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:33:58.800636 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:33:58.800643 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:33:58.800679 (+ 36us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:33:58.801057 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.801068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.802607 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.802612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.803898 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:33:58.803914 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.805005 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.805013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:33:58.805149 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.805234 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:33:58.805727 (+ 493us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:33:58.805736 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 00:33:58.805803 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 00:33:58.805930 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":340,"threads_started":1}
I20260812 00:33:58.816083 11511 consensus_queue.cc:1035] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56f888bce52442598fed56fa788073d2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 39957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:33:58.818414 10833 raft_consensus.cc:1261] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Refusing update from remote peer 97d30b7b13d44aa8bd90ef616138a026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:33:58.819054 11511 consensus_queue.cc:1035] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa9ac64c093141a5b099fdcfd1ec4dd0" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:33:58.898818 10462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:33:58.899127 10462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:33:58.902642 10462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81550dfc70ad4409b041664e6ee81beb: Sending DeleteTablet for 3 replicas of tablet abaa61c108e3410b86f5a9891d54e510
I20260812 00:33:58.903589 10774 tablet_service.cc:1514] Processing DeleteTablet for tablet abaa61c108e3410b86f5a9891d54e510 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:33:58 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:42214
I20260812 00:33:58.903657 11071 tablet_service.cc:1514] Processing DeleteTablet for tablet abaa61c108e3410b86f5a9891d54e510 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:33:58 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:35662
I20260812 00:33:58.904145 11333 tablet_service.cc:1514] Processing DeleteTablet for tablet abaa61c108e3410b86f5a9891d54e510 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:33:58 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:57408
I20260812 00:33:58.904322 11645 tablet_replica.cc:330] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: stopping tablet replica
I20260812 00:33:58.904556 11645 raft_consensus.cc:2229] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:33:58.904935 11645 raft_consensus.cc:2258] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:33:58.904999 11649 tablet_replica.cc:330] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: stopping tablet replica
I20260812 00:33:58.905378 11649 raft_consensus.cc:2229] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:33:58.906061 11649 raft_consensus.cc:2258] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:33:58.906913 11645 ts_tablet_manager.cc:1905] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:33:58.907223 11649 ts_tablet_manager.cc:1905] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:33:58.915076 11649 ts_tablet_manager.cc:1918] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:33:58.915095 11645 ts_tablet_manager.cc:1918] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:33:58.915256 11645 log.cc:1196] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/abaa61c108e3410b86f5a9891d54e510
I20260812 00:33:58.915477 11649 log.cc:1196] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/abaa61c108e3410b86f5a9891d54e510
I20260812 00:33:58.915804 11645 ts_tablet_manager.cc:1939] T abaa61c108e3410b86f5a9891d54e510 P aa9ac64c093141a5b099fdcfd1ec4dd0: Deleting consensus metadata
I20260812 00:33:58.916217 11649 ts_tablet_manager.cc:1939] T abaa61c108e3410b86f5a9891d54e510 P 97d30b7b13d44aa8bd90ef616138a026: Deleting consensus metadata
I20260812 00:33:58.917320 10434 catalog_manager.cc:4841] TS aa9ac64c093141a5b099fdcfd1ec4dd0 (127.7.244.65:41583): tablet abaa61c108e3410b86f5a9891d54e510 (table test-table [id=2636887fe0e345b296ea36a0d99a1e02]) successfully deleted
I20260812 00:33:58.918335 10430 catalog_manager.cc:4841] TS 97d30b7b13d44aa8bd90ef616138a026 (127.7.244.67:45503): tablet abaa61c108e3410b86f5a9891d54e510 (table test-table [id=2636887fe0e345b296ea36a0d99a1e02]) successfully deleted
I20260812 00:33:58.921459 11648 tablet_replica.cc:330] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: stopping tablet replica
I20260812 00:33:58.921744 11648 raft_consensus.cc:2229] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:33:58.922039 11648 raft_consensus.cc:2258] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:33:58.923502 11648 ts_tablet_manager.cc:1905] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:33:58.930430 11648 ts_tablet_manager.cc:1918] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:33:58.930615 11648 log.cc:1196] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/abaa61c108e3410b86f5a9891d54e510
I20260812 00:33:58.931198 11648 ts_tablet_manager.cc:1939] T abaa61c108e3410b86f5a9891d54e510 P 56f888bce52442598fed56fa788073d2: Deleting consensus metadata
I20260812 00:33:58.932421 10437 catalog_manager.cc:4841] TS 56f888bce52442598fed56fa788073d2 (127.7.244.66:39957): tablet abaa61c108e3410b86f5a9891d54e510 (table test-table [id=2636887fe0e345b296ea36a0d99a1e02]) successfully deleted
Aug 12 00:33:58 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:33:58.946354 11651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.918363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.7.244.65:41583)
0812 00:33:58.924870 (+ 6507us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.924878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.926065 (+ 1187us) server_negotiation.cc:407] Connection header received
0812 00:33:58.926307 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.926313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.926401 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.926513 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:58.928680 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.929979 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.931982 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.932676 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.932833 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.938204 (+ 5371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:58.938227 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:58.938248 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:58.938311 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:58.942595 (+ 4284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:58.943756 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:58.943766 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:58.943771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:58.943977 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:58.944467 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:58.944472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:58.944475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:58.944968 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:58.945284 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.945676 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.945962 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6348,"thread_start_us":6169,"threads_started":1}
W20260812 00:33:58.947310 10911 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42216
I20260812 00:33:58.971040 11586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.954762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.7.244.126:44849)
0812 00:33:58.955077 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.955084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.955665 (+ 581us) server_negotiation.cc:407] Connection header received
0812 00:33:58.955981 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.955988 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.956075 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.956149 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:58.958237 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.959464 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.961475 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.962193 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.962307 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.965225 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:58.965231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:58.965234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:58.965277 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:58.968082 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:58.968856 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:58.968863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:58.968865 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:58.969021 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:58.969482 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:58.969487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:58.969489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:58.969809 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:58.970076 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.970463 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.970724 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
W20260812 00:33:58.971741 10462 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39928
I20260812 00:33:58.973966 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 10634
I20260812 00:33:58.995065 11651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:58.975279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.7.244.65:41583)
0812 00:33:58.975472 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:58.975478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:58.980646 (+ 5168us) server_negotiation.cc:407] Connection header received
0812 00:33:58.980796 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:58.980803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:58.980930 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:58.981059 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:58.982439 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.983565 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.984804 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:58.985575 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:58.985728 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:58.988263 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:58.988271 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:58.988276 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:58.988328 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:58.991913 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:58.992731 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:58.992738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:58.992742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:58.992966 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:58.993375 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:58.993381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:58.993384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:58.993823 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:58.994120 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:58.994435 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 00:33:58.994698 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260812 00:33:58.995750 10911 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42228
W20260812 00:33:58.996416 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 10634: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:33:58.996673 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 10634
I20260812 00:33:59.070417 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 10933
Aug 12 00:33:59 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:33:59.090344 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:59.071355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.7.244.66:39957)
0812 00:33:59.071873 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:59.071878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:59.071896 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:33:59.071965 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:59.071969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:59.072031 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:59.072124 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:59.073832 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:59.074540 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:59.076479 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:59.076949 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:59.077058 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:59.082564 (+ 5506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:59.082592 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:59.082617 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:59.082679 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:59.086906 (+ 4227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:59.087843 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:59.087854 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:59.087858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:59.088073 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:59.088461 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:59.088466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:59.088470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:59.089006 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:59.089442 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:59.089642 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 00:33:59.089922 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":297,"threads_started":1}
W20260812 00:33:59.091130 11195 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35674
W20260812 00:33:59.091827 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 10933: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:33:59.092031 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 10933
I20260812 00:33:59.126092 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11209
Aug 12 00:33:59 dist-test-slave-gjw7 krb5kdc[10372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:33:59.144696 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:33:59.127001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.7.244.67:45503)
0812 00:33:59.127239 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 00:33:59.127243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:33:59.127263 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:33:59.127408 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:33:59.127412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:33:59.127484 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:33:59.127561 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:33:59.129181 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:59.130148 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:59.132026 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:33:59.132548 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:33:59.132659 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:33:59.138155 (+ 5496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:33:59.138171 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:33:59.138187 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:33:59.138237 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:33:59.141400 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:59.142329 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:59.142336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:59.142339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:59.142504 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:33:59.143085 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:33:59.143090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:33:59.143092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:33:59.143564 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:33:59.143816 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 00:33:59.144149 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 00:33:59.144375 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 00:33:59.145565 11469 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57428
W20260812 00:33:59.146342 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11209: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:33:59.146520 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 11209
I20260812 00:33:59.184026 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 10398
I20260812 00:33:59.185379 10559 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:33:59.301005 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 10398
2026-08-12T00:33:59Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5342 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:33:59 dist-test-slave-gjw7 krb5kdc[11669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:33:59 dist-test-slave-gjw7 krb5kdc[11669](info): set up 2 sockets
Aug 12 00:33:59 dist-test-slave-gjw7 krb5kdc[11669](info): commencing operation
krb5kdc: starting...
W20260812 00:34:00.608742 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.006s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:00 dist-test-slave-gjw7 krb5kdc[11669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:34:00.838711 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44065
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:44065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:01.206400 11685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:01.206737 11685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:01.206871 11685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:01.214649 11685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:01.214781 11685 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 00:34:01.214829 11685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:01.214923 11685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:01.214993 11685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:01.215075 11685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:01.225701 11685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:44065
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:01.227159 11685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:01.228678 11685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:01.242296 11700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:01.242380 11704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:01.242266 11701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:01.243110 11685 server_base.cc:1034] running on GCE node
I20260812 00:34:01.244059 11685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:01.245678 11685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:01.246963 11685 hybrid_clock.cc:648] HybridClock initialized: now 1786494841246868 us; error 67 us; skew 500 ppm
I20260812 00:34:01.247386 11685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:01 dist-test-slave-gjw7 krb5kdc[11669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:01.251183 11685 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:34:01.252974 11685 webserver.cc:460] Webserver started at http://127.7.244.126:36631/ using document root <none> and password file <none>
I20260812 00:34:01.253834 11685 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:01.253974 11685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:01.254328 11685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:01.257995 11685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "98eeb76b992e451f8842ea70656414e6"
format_stamp: "Formatted at 2026-08-12 00:34:01 on dist-test-slave-gjw7"
server_key: "0183149b51ccb52da8323e9f6a9ab59a"
server_key_iv: "d1f3beeafe2f9fddbbda0ab368b4089e"
server_key_version: "encryptionkey@0"
I20260812 00:34:01.258879 11685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "98eeb76b992e451f8842ea70656414e6"
format_stamp: "Formatted at 2026-08-12 00:34:01 on dist-test-slave-gjw7"
server_key: "0183149b51ccb52da8323e9f6a9ab59a"
server_key_iv: "d1f3beeafe2f9fddbbda0ab368b4089e"
server_key_version: "encryptionkey@0"
I20260812 00:34:01.265062 11685 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:34:01.269145 11719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:01.270835 11685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:34:01.271031 11685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "98eeb76b992e451f8842ea70656414e6"
format_stamp: "Formatted at 2026-08-12 00:34:01 on dist-test-slave-gjw7"
server_key: "0183149b51ccb52da8323e9f6a9ab59a"
server_key_iv: "d1f3beeafe2f9fddbbda0ab368b4089e"
server_key_version: "encryptionkey@0"
I20260812 00:34:01.271279 11685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:01.303030 11685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:01.308079 11685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:01.308432 11685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:01.332098 11685 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:44065
I20260812 00:34:01.332118 11838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:44065 every 8 connection(s)
I20260812 00:34:01.333765 11685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:01.336480 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 11685
I20260812 00:34:01.336720 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:01.337677 8145 external_mini_cluster.cc:1428] Setting key 2ba93eb17be69f07821814b540b09fb0
I20260812 00:34:01.338965 11839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 00:34:01 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:01.352922 11839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: Bootstrap starting.
I20260812 00:34:01.356848 11839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:01.358093 11839 log.cc:824] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:01.361438 11839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: No bootstrap required, opened a new log
I20260812 00:34:01.361826 11844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:01.340084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.7.244.126:44065)
0812 00:34:01.340959 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:01.340973 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:01.341017 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:34:01.342681 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:01.342708 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:01.343178 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:01.343601 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:01.345170 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:01.346642 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:01.348043 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:01.348814 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:01.349033 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:01.353653 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:01.353667 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:01.353693 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:01.353746 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:01.357151 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:01.358050 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:01.358057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:01.358065 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:01.358296 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:01.358723 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:01.358727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:01.358729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:01.359377 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:01.359701 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:01.360175 (+ 474us) server_negotiation.cc:299] Negotiation successful
0812 00:34:01.360644 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":391,"threads_started":1}
I20260812 00:34:01.370091 11839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eeb76b992e451f8842ea70656414e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44065 } }
I20260812 00:34:01.370575 11839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:01.370688 11839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98eeb76b992e451f8842ea70656414e6, State: Initialized, Role: FOLLOWER
I20260812 00:34:01.371302 11839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eeb76b992e451f8842ea70656414e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44065 } }
I20260812 00:34:01.371546 11839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:01.371675 11839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:01.371845 11839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:01.374284 11839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eeb76b992e451f8842ea70656414e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44065 } }
I20260812 00:34:01.374894 11839 leader_election.cc:302] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98eeb76b992e451f8842ea70656414e6; no voters:
I20260812 00:34:01.375837 11839 leader_election.cc:288] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:01.375833 11846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:01.377194 11846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [term 1 LEADER]: Becoming Leader. State: Replica: 98eeb76b992e451f8842ea70656414e6, State: Running, Role: LEADER
I20260812 00:34:01.377797 11846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eeb76b992e451f8842ea70656414e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44065 } }
I20260812 00:34:01.378281 11839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:01.380513 11847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98eeb76b992e451f8842ea70656414e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eeb76b992e451f8842ea70656414e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44065 } } }
I20260812 00:34:01.380963 11847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:01.381767 11848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98eeb76b992e451f8842ea70656414e6. Latest consensus state: current_term: 1 leader_uuid: "98eeb76b992e451f8842ea70656414e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98eeb76b992e451f8842ea70656414e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44065 } } }
I20260812 00:34:01.382062 11848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:01.383682 11851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:01.389688 11851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:01.400897 11851 catalog_manager.cc:1347] Generated new cluster ID: b0e36933ec274cf78d35f169694b87d6
I20260812 00:34:01.401042 11851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:01.415235 11851 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:01.417369 11851 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:01.437466 11851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: Generated new TSK 0
I20260812 00:34:01.438630 11851 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:34:01.503038 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44065
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:01.862651 11886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:01.862993 11886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:01.863143 11886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:01.870836 11886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:01.870994 11886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:01.871117 11886 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 00:34:01.871184 11886 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:34:01.871292 11886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:01.883778 11886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:44065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:01.886077 11886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:01.888182 11886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:01.904047 11905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:01.904086 11896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:01.904109 11900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:01.905004 11886 server_base.cc:1034] running on GCE node
I20260812 00:34:01.905809 11886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:01.906917 11886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:01.908145 11886 hybrid_clock.cc:648] HybridClock initialized: now 1786494841908081 us; error 57 us; skew 500 ppm
I20260812 00:34:01.908509 11886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:01 dist-test-slave-gjw7 krb5kdc[11669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:01.917237 11886 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:34:01.919081 11886 webserver.cc:460] Webserver started at http://127.7.244.65:43065/ using document root <none> and password file <none>
I20260812 00:34:01.919940 11886 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:01.920069 11886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:01.920380 11886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:01.923714 11886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "97b6a059e0a24df3acacc16cbe10733e"
format_stamp: "Formatted at 2026-08-12 00:34:01 on dist-test-slave-gjw7"
server_key: "5dbc3dee8af29f7bfe29b81654bc3dc6"
server_key_iv: "53ee95ce9a1a0ddc7fd88333363dfc36"
server_key_version: "encryptionkey@0"
I20260812 00:34:01.924597 11886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "97b6a059e0a24df3acacc16cbe10733e"
format_stamp: "Formatted at 2026-08-12 00:34:01 on dist-test-slave-gjw7"
server_key: "5dbc3dee8af29f7bfe29b81654bc3dc6"
server_key_iv: "53ee95ce9a1a0ddc7fd88333363dfc36"
server_key_version: "encryptionkey@0"
I20260812 00:34:01.930841 11886 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260812 00:34:01.935105 11914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:01.936913 11886 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260812 00:34:01.937115 11886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "97b6a059e0a24df3acacc16cbe10733e"
format_stamp: "Formatted at 2026-08-12 00:34:01 on dist-test-slave-gjw7"
server_key: "5dbc3dee8af29f7bfe29b81654bc3dc6"
server_key_iv: "53ee95ce9a1a0ddc7fd88333363dfc36"
server_key_version: "encryptionkey@0"
I20260812 00:34:01.937389 11886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:01.955302 11886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:01.960129 11886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:01.960444 11886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:01.963567 11886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:01.963696 11886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:01.963829 11886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:01.963923 11886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:34:01 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:01.992022 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:01.964534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:47025 (local address 127.7.244.126:44065)
0812 00:34:01.965018 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:01.965022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:01.966508 (+ 1486us) server_negotiation.cc:407] Connection header received
0812 00:34:01.968713 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:01.968719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:01.968822 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:01.968921 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:01.971521 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:01.972527 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:01.976021 (+ 3494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:01.976718 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:01.976859 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:01.983475 (+ 6616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:01.983485 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:01.983492 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:01.983554 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:01.986938 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:01.988365 (+ 1427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:01.988374 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:01.988378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:01.988572 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:01.989212 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:01.989219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:01.989222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:01.989701 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:01.989869 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:01.991112 (+ 1243us) server_negotiation.cc:299] Negotiation successful
0812 00:34:01.991538 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":247,"threads_started":1}
I20260812 00:34:01.992705 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:01.965324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44065 (local address 127.7.244.65:47025)
0812 00:34:01.966290 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:01.966360 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:01.968364 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:01.969266 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:01.969280 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:01.970204 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:01.971251 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:01.971295 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:01.972792 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:01.972798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:01.975699 (+ 2901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:01.975728 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:01.977427 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:01.977441 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:01.977977 (+ 536us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:01.979119 (+ 1142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:01.979152 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:01.983017 (+ 3865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:01.987166 (+ 4149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:01.987183 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:01.987214 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:01.988166 (+ 952us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:01.988695 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:01.988701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:01.988705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:01.989040 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:01.989873 (+ 833us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:01.989883 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:01.990475 (+ 592us) client_negotiation.cc:769] Sending connection context
0812 00:34:01.990900 (+ 425us) client_negotiation.cc:240] Negotiation successful
0812 00:34:01.991389 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":382,"threads_started":1}
I20260812 00:34:02.014148 11886 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:39049
I20260812 00:34:02.014192 12171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:39049 every 8 connection(s)
I20260812 00:34:02.014981 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:01.999160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:44151 (local address 127.7.244.126:44065)
0812 00:34:01.999392 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:01.999400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:01.999942 (+ 542us) server_negotiation.cc:407] Connection header received
0812 00:34:02.000112 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:02.000118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:02.000218 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:02.000355 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:02.001572 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.002455 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.004421 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.004899 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.005026 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:02.007832 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:02.007840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:02.007845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:02.007896 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:02.011600 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:02.012556 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:02.012561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:02.012563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:02.012721 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:02.013189 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:02.013194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:02.013197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:02.013618 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:02.013788 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:02.014340 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 00:34:02.014601 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 00:34:02.015081 12115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:01.999169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44065 (local address 127.7.244.65:44151)
0812 00:34:01.999822 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:01.999847 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:01.999990 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:02.000458 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:02.000462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:02.001009 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:02.001383 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:02.001398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.002621 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.002625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:02.004221 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:02.004242 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.005121 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.005129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:02.005353 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:02.006290 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:02.006320 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:02.007556 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:02.011865 (+ 4309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:02.011870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:02.011873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:02.012372 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:02.012855 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:02.012860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:02.012864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:02.013025 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:02.013759 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:02.013764 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:02.014046 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 00:34:02.014193 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 00:34:02.014601 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":341,"threads_started":1}
I20260812 00:34:02.015964 11886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:02.018373 12172 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44065
I20260812 00:34:02.018643 12172 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:02.019397 12172 heartbeater.cc:511] Master 127.7.244.126:44065 requested a full tablet report, sending...
I20260812 00:34:02.021719 11747 ts_manager.cc:194] Registered new tserver with Master: 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049)
I20260812 00:34:02.021862 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 11886
I20260812 00:34:02.022070 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:02.022897 8145 external_mini_cluster.cc:1428] Setting key 779617c4a0d8b551d403923c7e9617ec
I20260812 00:34:02.023864 11747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:47025
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:34:02.073642 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44065
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:02.425810 12179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:02.426111 12179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:02.426266 12179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:02.433898 12179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:02.434038 12179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:02.434222 12179 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 00:34:02.434309 12179 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:34:02.434402 12179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:02.445043 12179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:44065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:02.446555 12179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:02.447955 12179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:02.462069 12196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:02.462069 12199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:02.462707 12195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:02.471292 12179 server_base.cc:1034] running on GCE node
I20260812 00:34:02.472098 12179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:02.473274 12179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:02.474606 12179 hybrid_clock.cc:648] HybridClock initialized: now 1786494842474521 us; error 65 us; skew 500 ppm
I20260812 00:34:02.475046 12179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:02 dist-test-slave-gjw7 krb5kdc[11669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494842, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:02.479146 12179 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:34:02.481004 12179 webserver.cc:460] Webserver started at http://127.7.244.66:36073/ using document root <none> and password file <none>
I20260812 00:34:02.481905 12179 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:02.482056 12179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:02.482437 12179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:02.485929 12179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "17961f9df85a492db1d4aaa5b20089a3"
format_stamp: "Formatted at 2026-08-12 00:34:02 on dist-test-slave-gjw7"
server_key: "22174e00d0f94a9ab977c831c09abca4"
server_key_iv: "b0c5faa1e2176c2e46239782cbd27b09"
server_key_version: "encryptionkey@0"
I20260812 00:34:02.486804 12179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "17961f9df85a492db1d4aaa5b20089a3"
format_stamp: "Formatted at 2026-08-12 00:34:02 on dist-test-slave-gjw7"
server_key: "22174e00d0f94a9ab977c831c09abca4"
server_key_iv: "b0c5faa1e2176c2e46239782cbd27b09"
server_key_version: "encryptionkey@0"
I20260812 00:34:02.492750 12179 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:34:02.497076 12218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:02.498690 12179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 00:34:02.498873 12179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "17961f9df85a492db1d4aaa5b20089a3"
format_stamp: "Formatted at 2026-08-12 00:34:02 on dist-test-slave-gjw7"
server_key: "22174e00d0f94a9ab977c831c09abca4"
server_key_iv: "b0c5faa1e2176c2e46239782cbd27b09"
server_key_version: "encryptionkey@0"
I20260812 00:34:02.499100 12179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:02.524420 12179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:02.529757 12179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:02.530105 12179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:02.533324 12179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:02.533475 12179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 00:34:02.533612 12179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:02.533718 12179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:34:02 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494842, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:02.564292 12234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:02.534812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44065 (local address 127.7.244.66:39885)
0812 00:34:02.535816 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:02.535892 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:02.538877 (+ 2985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:02.540033 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:02.540048 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:02.540958 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:02.542012 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:02.542050 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.543832 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.543839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:02.545927 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:02.545948 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.547311 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.547318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:02.547736 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:02.548778 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:02.548846 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:02.553414 (+ 4568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:02.558158 (+ 4744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:02.558173 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:02.558197 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:02.559202 (+ 1005us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:02.559820 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:02.559827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:02.559831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:02.560125 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:02.560890 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:02.560905 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:02.561542 (+ 637us) client_negotiation.cc:769] Sending connection context
0812 00:34:02.562031 (+ 489us) client_negotiation.cc:240] Negotiation successful
0812 00:34:02.562770 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":371,"threads_started":1}
I20260812 00:34:02.565579 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:02.534003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:39885 (local address 127.7.244.126:44065)
0812 00:34:02.534678 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:02.534685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:02.536085 (+ 1400us) server_negotiation.cc:407] Connection header received
0812 00:34:02.539376 (+ 3291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:02.539383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:02.539492 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:02.540500 (+ 1008us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:02.542280 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.543583 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.546141 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.546772 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.546946 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:02.553871 (+ 6925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:02.553881 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:02.553887 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:02.553947 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:02.557882 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:02.559430 (+ 1548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:02.559437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:02.559441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:02.559662 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:02.560343 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:02.560350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:02.560353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:02.560723 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:02.560912 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:02.565002 (+ 4090us) server_negotiation.cc:299] Negotiation successful
0812 00:34:02.565246 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":415,"threads_started":1}
I20260812 00:34:02.584753 12179 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:33591
I20260812 00:34:02.584812 12438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:33591 every 8 connection(s)
I20260812 00:34:02.586534 12179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:02.587841 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 12179
I20260812 00:34:02.588034 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:02.588903 8145 external_mini_cluster.cc:1428] Setting key 083d642afad360b0935de21beab0968e
I20260812 00:34:02.589186 12439 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44065
I20260812 00:34:02.589557 12439 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:02.591355 12439 heartbeater.cc:511] Master 127.7.244.126:44065 requested a full tablet report, sending...
I20260812 00:34:02.593127 11745 ts_manager.cc:194] Registered new tserver with Master: 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591)
I20260812 00:34:02.593580 12404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:02.576608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44065 (local address 127.7.244.66:33799)
0812 00:34:02.577478 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:02.577503 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:02.577695 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:02.578095 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:02.578100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:02.578758 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:02.579160 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:02.579176 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.580747 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.580753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:02.582425 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:02.582446 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.583536 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.583549 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:02.583728 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:02.584560 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:02.584579 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:02.585749 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:02.590538 (+ 4789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:02.590545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:02.590549 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:02.591161 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:02.591623 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:02.591628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:02.591633 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:02.591779 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:02.592444 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:02.592449 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:02.592627 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 00:34:02.592767 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 00:34:02.593122 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":473,"threads_started":1}
I20260812 00:34:02.594435 11745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:39885
I20260812 00:34:02.594656 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:02.576509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:33799 (local address 127.7.244.126:44065)
0812 00:34:02.576713 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:02.576719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:02.577621 (+ 902us) server_negotiation.cc:407] Connection header received
0812 00:34:02.577829 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:02.577832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:02.577919 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:02.578024 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:02.579360 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.580551 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.582615 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:02.583224 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:02.583384 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:02.586002 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:02.586010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:02.586016 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:02.586067 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:02.590353 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:02.591345 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:02.591349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:02.591352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:02.591503 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:02.591904 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:02.591909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:02.591913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:02.592297 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:02.593873 (+ 1576us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:02.594017 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 00:34:02.594295 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:34:02.637516 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44065
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:02.984154 12446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:02.984462 12446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:02.984601 12446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:02.992041 12446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:02.992177 12446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:02.992271 12446 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 00:34:02.992347 12446 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:34:02.992455 12446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:03.002756 12446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:44065
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:03.004259 12446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:03.005781 12446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:03.020285 12463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:03.020362 12466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:03.020320 12464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:03.021602 12446 server_base.cc:1034] running on GCE node
I20260812 00:34:03.022393 12446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:03.023466 12446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:03.024739 12446 hybrid_clock.cc:648] HybridClock initialized: now 1786494843024672 us; error 54 us; skew 500 ppm
I20260812 00:34:03.025131 12446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:03.027071 12172 heartbeater.cc:503] Master 127.7.244.126:44065 was elected leader, sending a full tablet report...
Aug 12 00:34:03 dist-test-slave-gjw7 krb5kdc[11669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494843, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:03.028966 12446 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:34:03.030731 12446 webserver.cc:460] Webserver started at http://127.7.244.67:34469/ using document root <none> and password file <none>
I20260812 00:34:03.031494 12446 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:03.031628 12446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:03.031971 12446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:03.035252 12446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "9ae41a64a78145c9af71c25668989429"
format_stamp: "Formatted at 2026-08-12 00:34:03 on dist-test-slave-gjw7"
server_key: "641f7907e150ae5059fc6c02a3ec9a9a"
server_key_iv: "1b75f23f884735837c2a9072718a6e0d"
server_key_version: "encryptionkey@0"
I20260812 00:34:03.036147 12446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "9ae41a64a78145c9af71c25668989429"
format_stamp: "Formatted at 2026-08-12 00:34:03 on dist-test-slave-gjw7"
server_key: "641f7907e150ae5059fc6c02a3ec9a9a"
server_key_iv: "1b75f23f884735837c2a9072718a6e0d"
server_key_version: "encryptionkey@0"
I20260812 00:34:03.042014 12446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 00:34:03.046057 12484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:03.047560 12446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:34:03.047752 12446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "9ae41a64a78145c9af71c25668989429"
format_stamp: "Formatted at 2026-08-12 00:34:03 on dist-test-slave-gjw7"
server_key: "641f7907e150ae5059fc6c02a3ec9a9a"
server_key_iv: "1b75f23f884735837c2a9072718a6e0d"
server_key_version: "encryptionkey@0"
I20260812 00:34:03.047964 12446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:03.067956 12446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:03.073118 12446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:03.073503 12446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:03.076776 12446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:03.076905 12446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:03.077033 12446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:03.077108 12446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:34:03 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494843, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:03.105180 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.078113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44065 (local address 127.7.244.67:55355)
0812 00:34:03.079073 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.079162 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.081953 (+ 2791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.083085 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.083102 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.084112 (+ 1010us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:03.085166 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.085206 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.086947 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.086954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.089695 (+ 2741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.089725 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.090930 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.090944 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.091218 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.092002 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:03.092028 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:03.095208 (+ 3180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:03.099813 (+ 4605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:03.099825 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:03.099846 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:03.100440 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:03.100936 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:03.100941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:03.100943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:03.101159 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:03.101901 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:03.101915 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:03.102396 (+ 481us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.102784 (+ 388us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.103435 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":327,"threads_started":1}
I20260812 00:34:03.106482 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.077443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:55355 (local address 127.7.244.126:44065)
0812 00:34:03.077650 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.077656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.079306 (+ 1650us) server_negotiation.cc:407] Connection header received
0812 00:34:03.082481 (+ 3175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.082488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.082588 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.082701 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:03.085448 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.086665 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.089961 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.090603 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.090777 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.095579 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:03.095585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:03.095589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:03.095629 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:03.099553 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:03.100589 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:03.100595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:03.100599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:03.100804 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:03.101272 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:03.101278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:03.101282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:03.101718 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:03.101881 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.105944 (+ 4063us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.106211 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:34:03.128198 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.112074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:44135 (local address 127.7.244.126:44065)
0812 00:34:03.112274 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.112281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.113097 (+ 816us) server_negotiation.cc:407] Connection header received
0812 00:34:03.113384 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.113390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.113577 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.113673 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:03.115292 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.116387 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.118408 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.118980 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.119125 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.121896 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:03.121904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:03.121909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:03.121960 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:03.125340 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:03.126135 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:03.126140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:03.126142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:03.126316 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:03.126708 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:03.126714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:03.126717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:03.127022 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:03.127164 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.127595 (+ 431us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.127832 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:34:03.128297 12640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.112122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44065 (local address 127.7.244.67:44135)
0812 00:34:03.112951 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.112979 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.113160 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.113948 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.113953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.114719 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:03.115120 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.115137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.116548 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.116558 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.118226 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.118256 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.119253 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.119262 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.119484 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.120246 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:03.120273 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:03.121559 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:03.125507 (+ 3948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:03.125513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:03.125518 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:03.125989 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:03.126438 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:03.126444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:03.126448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:03.126593 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:03.127146 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:03.127151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:03.127343 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.127469 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.127868 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":454,"threads_started":1}
I20260812 00:34:03.137008 12446 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:42211
I20260812 00:34:03.137069 12729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:42211 every 8 connection(s)
I20260812 00:34:03.138427 12446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:03.140515 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 12446
I20260812 00:34:03.140710 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:03.140981 12730 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44065
I20260812 00:34:03.141268 12730 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:03.141360 8145 external_mini_cluster.cc:1428] Setting key 4e35532dcb7a847a73d6462889c6b0b0
I20260812 00:34:03.142215 12730 heartbeater.cc:511] Master 127.7.244.126:44065 requested a full tablet report, sending...
I20260812 00:34:03.143868 11745 ts_manager.cc:194] Registered new tserver with Master: 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211)
I20260812 00:34:03.144733 11745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:55355
I20260812 00:34:03.144855 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:34:03 dist-test-slave-gjw7 krb5kdc[11669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:34:03 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:03.200651 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.185441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42680 (local address 127.7.244.126:44065)
0812 00:34:03.185658 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.185663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.185767 (+ 104us) server_negotiation.cc:407] Connection header received
0812 00:34:03.185934 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.185938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.186009 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.186101 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:03.187707 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.188527 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.189910 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.190405 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.190527 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.194699 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:03.194705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:03.194708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:03.194755 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:03.197662 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:03.198536 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:03.198541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:03.198543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:03.198696 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:03.199120 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:03.199124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:03.199126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:03.199426 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:03.199641 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.200017 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.200275 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:34:03.205933 11745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:03.213223 11745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:03.240736 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.230136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.7.244.67:42211)
0812 00:34:03.232649 (+ 2513us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.232661 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.232701 (+ 40us) server_negotiation.cc:407] Connection header received
0812 00:34:03.232827 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.232835 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.233090 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.234097 (+ 1007us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:03.234461 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.235722 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.238010 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.239466 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.239599 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.239841 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.240204 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.240382 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2310,"thread_start_us":2074,"threads_started":1}
I20260812 00:34:03.240808 12765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.230558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:42211 (local address 127.0.0.1:53742)
0812 00:34:03.231734 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.231760 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.231917 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.233396 (+ 1479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.233403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.233436 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:03.233879 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.233891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.235936 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.235942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.237802 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.237818 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.239710 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.239717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.239895 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.239967 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.240127 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.240341 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1038,"mutex_wait_us":1,"thread_start_us":891,"threads_started":1}
I20260812 00:34:03.240852 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.226936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:33591 (local address 127.0.0.1:55942)
0812 00:34:03.231371 (+ 4435us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.231400 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.231668 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.232741 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.232746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.232774 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:03.233083 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.233092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.234947 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.234952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.237334 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.237350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.239576 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.239583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.240176 (+ 593us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.240212 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.240311 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.240442 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4272,"mutex_wait_us":4,"thread_start_us":1908,"threads_started":1}
I20260812 00:34:03.241617 12762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.227273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.7.244.66:33591)
0812 00:34:03.228672 (+ 1399us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.228685 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.231528 (+ 2843us) server_negotiation.cc:407] Connection header received
0812 00:34:03.231922 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.231934 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.232282 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.232472 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:03.233270 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.234746 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.237535 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.238991 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.239141 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.239399 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.241113 (+ 1714us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.241341 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1194,"thread_start_us":1037,"threads_started":1}
I20260812 00:34:03.242923 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.226630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:39049 (local address 127.0.0.1:39774)
0812 00:34:03.229228 (+ 2598us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.229282 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.229490 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.235721 (+ 6231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.235726 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.235755 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:03.236342 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.236356 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.238138 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.238143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.240061 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.240075 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.242088 (+ 2013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.242095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.242272 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.242319 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.242416 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.242558 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2357,"mutex_wait_us":3,"thread_start_us":4396,"threads_started":1}
I20260812 00:34:03.243728 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.226823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39774 (local address 127.7.244.65:39049)
0812 00:34:03.235059 (+ 8236us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.235068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.235101 (+ 33us) server_negotiation.cc:407] Connection header received
0812 00:34:03.235214 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.235224 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.235483 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.235699 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:03.236541 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.237927 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.240246 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.241675 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.241808 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.242007 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.243249 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.243442 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8049,"thread_start_us":7903,"threads_started":1}
I20260812 00:34:03.245481 12313 tablet_service.cc:1467] Processing CreateTablet for tablet 28e056e5d796485caf7807e2cf74854a (DEFAULT_TABLE table=test-table [id=7517dcb335ac4ef7850ccf702ddcce2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:03.245502 12590 tablet_service.cc:1467] Processing CreateTablet for tablet 28e056e5d796485caf7807e2cf74854a (DEFAULT_TABLE table=test-table [id=7517dcb335ac4ef7850ccf702ddcce2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:03.246866 12313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e056e5d796485caf7807e2cf74854a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.247042 12590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e056e5d796485caf7807e2cf74854a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.248026 12034 tablet_service.cc:1467] Processing CreateTablet for tablet 28e056e5d796485caf7807e2cf74854a (DEFAULT_TABLE table=test-table [id=7517dcb335ac4ef7850ccf702ddcce2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:03.249769 12034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e056e5d796485caf7807e2cf74854a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.260179 12766 tablet_bootstrap.cc:492] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Bootstrap starting.
I20260812 00:34:03.263585 12766 tablet_bootstrap.cc:654] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.264628 12768 tablet_bootstrap.cc:492] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Bootstrap starting.
I20260812 00:34:03.265079 12766 log.cc:824] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:03.268553 12770 tablet_bootstrap.cc:492] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Bootstrap starting.
I20260812 00:34:03.268815 12768 tablet_bootstrap.cc:654] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.269150 12766 tablet_bootstrap.cc:492] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: No bootstrap required, opened a new log
I20260812 00:34:03.269528 12766 ts_tablet_manager.cc:1397] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260812 00:34:03.270468 12768 log.cc:824] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:03.272235 12770 tablet_bootstrap.cc:654] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.273653 12770 log.cc:824] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:03.274154 12768 tablet_bootstrap.cc:492] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: No bootstrap required, opened a new log
I20260812 00:34:03.274513 12768 ts_tablet_manager.cc:1397] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260812 00:34:03.277518 12770 tablet_bootstrap.cc:492] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: No bootstrap required, opened a new log
I20260812 00:34:03.277854 12770 ts_tablet_manager.cc:1397] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 00:34:03.279306 12766 raft_consensus.cc:348] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.280010 12766 raft_consensus.cc:374] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.280123 12766 raft_consensus.cc:729] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17961f9df85a492db1d4aaa5b20089a3, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.280819 12766 consensus_queue.cc:260] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.282092 12439 heartbeater.cc:503] Master 127.7.244.126:44065 was elected leader, sending a full tablet report...
I20260812 00:34:03.282816 12766 ts_tablet_manager.cc:1428] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260812 00:34:03.284925 12768 raft_consensus.cc:348] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.285542 12768 raft_consensus.cc:374] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.285679 12768 raft_consensus.cc:729] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae41a64a78145c9af71c25668989429, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.286340 12768 consensus_queue.cc:260] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.288580 12768 ts_tablet_manager.cc:1428] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260812 00:34:03.288726 12730 heartbeater.cc:503] Master 127.7.244.126:44065 was elected leader, sending a full tablet report...
I20260812 00:34:03.289989 12770 raft_consensus.cc:348] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.290519 12770 raft_consensus.cc:374] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.290652 12770 raft_consensus.cc:729] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b6a059e0a24df3acacc16cbe10733e, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.291385 12770 consensus_queue.cc:260] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.293388 12770 ts_tablet_manager.cc:1428] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260812 00:34:03.341595 12441 tablet.cc:2369] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:34:03.390184 12733 tablet.cc:2369] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:34:03.518955 12175 tablet.cc:2369] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:03.524494 12784 raft_consensus.cc:482] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:03.524915 12784 raft_consensus.cc:504] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.526772 12784 leader_election.cc:288] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591), 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211)
I20260812 00:34:03.535341 12762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.527166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:54373 (local address 127.7.244.66:33591)
0812 00:34:03.527356 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.527361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.527998 (+ 637us) server_negotiation.cc:407] Connection header received
0812 00:34:03.528232 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.528238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.528357 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.528470 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:03.529153 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.530241 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.532878 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.534124 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.534307 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.534387 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.534915 (+ 528us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.535055 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 00:34:03.535331 12763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.528281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:58529 (local address 127.7.244.67:42211)
0812 00:34:03.528494 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.528500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.528718 (+ 218us) server_negotiation.cc:407] Connection header received
0812 00:34:03.528889 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.528893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.528948 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.529018 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:03.529559 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.530507 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.532878 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.534124 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.534307 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.534387 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.534914 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.535055 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 00:34:03.535528 12788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.527144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:33591 (local address 127.7.244.65:54373)
0812 00:34:03.527890 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.527911 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.528064 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.528572 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.528577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.528605 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:03.529006 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.529020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.530416 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.530420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.532668 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.532685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.534461 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.534467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.534697 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.534756 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.534864 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.535060 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":69,"thread_start_us":363,"threads_started":1}
I20260812 00:34:03.535660 12789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.528014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:42211 (local address 127.7.244.65:58529)
0812 00:34:03.528624 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.528640 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.528780 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.529063 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.529067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.529094 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:03.529409 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.529419 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.530645 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.530650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.532685 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.532696 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.534448 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.534455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.534704 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.534766 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.534864 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.535060 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":64,"thread_start_us":314,"threads_started":1}
I20260812 00:34:03.536583 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e056e5d796485caf7807e2cf74854a" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3" is_pre_election: true
I20260812 00:34:03.536638 12653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e056e5d796485caf7807e2cf74854a" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae41a64a78145c9af71c25668989429" is_pre_election: true
I20260812 00:34:03.537108 12356 raft_consensus.cc:2454] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 0.
I20260812 00:34:03.537107 12653 raft_consensus.cc:2454] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 0.
I20260812 00:34:03.537798 11918 leader_election.cc:302] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b6a059e0a24df3acacc16cbe10733e, 9ae41a64a78145c9af71c25668989429; no voters:
I20260812 00:34:03.538235 12784 raft_consensus.cc:2781] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:03.538404 12784 raft_consensus.cc:482] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:03.538506 12784 raft_consensus.cc:3037] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:03.540812 12784 raft_consensus.cc:504] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.541555 12784 leader_election.cc:288] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 election: Requested vote from peers 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591), 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211)
I20260812 00:34:03.542201 12653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e056e5d796485caf7807e2cf74854a" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae41a64a78145c9af71c25668989429"
I20260812 00:34:03.542196 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e056e5d796485caf7807e2cf74854a" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3"
I20260812 00:34:03.542405 12356 raft_consensus.cc:3037] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:03.542420 12653 raft_consensus.cc:3037] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:03.544682 12653 raft_consensus.cc:2454] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 1.
I20260812 00:34:03.544682 12356 raft_consensus.cc:2454] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 1.
I20260812 00:34:03.545138 11918 leader_election.cc:302] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17961f9df85a492db1d4aaa5b20089a3, 97b6a059e0a24df3acacc16cbe10733e; no voters:
I20260812 00:34:03.545471 12784 raft_consensus.cc:2781] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:03.545928 12784 raft_consensus.cc:686] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 1 LEADER]: Becoming Leader. State: Replica: 97b6a059e0a24df3acacc16cbe10733e, State: Running, Role: LEADER
I20260812 00:34:03.546438 12784 consensus_queue.cc:237] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.551338 11745 catalog_manager.cc:5495] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65). New cstate: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } health_report { overall_health: UNKNOWN } } }
Aug 12 00:34:03 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494841, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:03.641993 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.628039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44065 (local address 127.0.0.1:42696)
0812 00:34:03.628786 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.628803 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.628957 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.629254 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.629258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.630021 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:03.630303 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.630315 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.631308 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.631313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.632333 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.632346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.633100 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.633107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.633271 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.634009 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:03.634033 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:03.636344 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:03.639282 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:03.639298 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:03.639305 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:03.639786 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:03.640146 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:03.640150 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:03.640152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:03.640335 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:03.640877 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:03.640888 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:03.641114 (+ 226us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.641285 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.641599 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":398,"threads_started":1}
I20260812 00:34:03.642010 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.628024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42696 (local address 127.7.244.126:44065)
0812 00:34:03.628292 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.628299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.628891 (+ 592us) server_negotiation.cc:407] Connection header received
0812 00:34:03.629070 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.629074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.629129 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.629217 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:03.630453 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.631155 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.632481 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.632909 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.633031 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.636562 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:03.636568 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:03.636571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:03.636604 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:03.639076 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:03.639903 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:03.639907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:03.639909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:03.640045 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:03.640430 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:03.640433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:03.640435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:03.640714 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:03.640866 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.641498 (+ 632us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.641743 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":66}
I20260812 00:34:03.647580 11745 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:42696:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 00:34:03.648586 11745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:03.655912 12313 tablet_service.cc:1467] Processing CreateTablet for tablet 9c475a89dd89445ab2b23dbdbe6c3329 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d996108858c49a387e2fa23aafc92e4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:34:03.655763 12034 tablet_service.cc:1467] Processing CreateTablet for tablet 9c475a89dd89445ab2b23dbdbe6c3329 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d996108858c49a387e2fa23aafc92e4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:34:03.656301 12590 tablet_service.cc:1467] Processing CreateTablet for tablet 9c475a89dd89445ab2b23dbdbe6c3329 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d996108858c49a387e2fa23aafc92e4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:34:03.656904 12313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c475a89dd89445ab2b23dbdbe6c3329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.656915 12034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c475a89dd89445ab2b23dbdbe6c3329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.657096 12590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c475a89dd89445ab2b23dbdbe6c3329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.663239 12766 tablet_bootstrap.cc:492] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: Bootstrap starting.
I20260812 00:34:03.664345 12770 tablet_bootstrap.cc:492] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: Bootstrap starting.
I20260812 00:34:03.664927 12768 tablet_bootstrap.cc:492] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: Bootstrap starting.
I20260812 00:34:03.664898 12766 tablet_bootstrap.cc:654] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.666025 12770 tablet_bootstrap.cc:654] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.666427 12768 tablet_bootstrap.cc:654] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.667647 12766 tablet_bootstrap.cc:492] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: No bootstrap required, opened a new log
I20260812 00:34:03.667802 12766 ts_tablet_manager.cc:1397] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 00:34:03.668481 12768 tablet_bootstrap.cc:492] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: No bootstrap required, opened a new log
I20260812 00:34:03.668637 12768 ts_tablet_manager.cc:1397] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260812 00:34:03.668486 12766 raft_consensus.cc:348] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.668710 12766 raft_consensus.cc:374] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.668804 12766 raft_consensus.cc:729] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17961f9df85a492db1d4aaa5b20089a3, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.669042 12766 consensus_queue.cc:260] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.669426 12768 raft_consensus.cc:348] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.669624 12768 raft_consensus.cc:374] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.669715 12768 raft_consensus.cc:729] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae41a64a78145c9af71c25668989429, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.669719 12766 ts_tablet_manager.cc:1428] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:34:03.669503 12773 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.669924 12773 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:03.669934 12768 consensus_queue.cc:260] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.670516 12768 ts_tablet_manager.cc:1428] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 00:34:03.670379 12774 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.670691 12774 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:03.671921 12770 tablet_bootstrap.cc:492] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: No bootstrap required, opened a new log
I20260812 00:34:03.672056 12770 ts_tablet_manager.cc:1397] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260812 00:34:03.672941 12770 raft_consensus.cc:348] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.673146 12770 raft_consensus.cc:374] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.673231 12770 raft_consensus.cc:729] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b6a059e0a24df3acacc16cbe10733e, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.673488 12770 consensus_queue.cc:260] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.673884 12784 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.674067 12770 ts_tablet_manager.cc:1428] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:34:03.674161 12784 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:03.714890 12784 raft_consensus.cc:482] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:03.715135 12784 raft_consensus.cc:504] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.715883 12784 leader_election.cc:288] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591), 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211)
I20260812 00:34:03.716521 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c475a89dd89445ab2b23dbdbe6c3329" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3" is_pre_election: true
I20260812 00:34:03.716588 12653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c475a89dd89445ab2b23dbdbe6c3329" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae41a64a78145c9af71c25668989429" is_pre_election: true
I20260812 00:34:03.716784 12356 raft_consensus.cc:2454] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 0.
I20260812 00:34:03.716854 12653 raft_consensus.cc:2454] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 0.
I20260812 00:34:03.717248 11918 leader_election.cc:302] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17961f9df85a492db1d4aaa5b20089a3, 97b6a059e0a24df3acacc16cbe10733e; no voters:
I20260812 00:34:03.717629 12784 raft_consensus.cc:2781] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:03.717741 12784 raft_consensus.cc:482] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:03.717852 12784 raft_consensus.cc:3037] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:03.719971 12784 raft_consensus.cc:504] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.720549 12784 leader_election.cc:288] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 election: Requested vote from peers 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591), 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211)
I20260812 00:34:03.721087 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c475a89dd89445ab2b23dbdbe6c3329" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3"
I20260812 00:34:03.721153 12653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c475a89dd89445ab2b23dbdbe6c3329" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae41a64a78145c9af71c25668989429"
I20260812 00:34:03.721341 12356 raft_consensus.cc:3037] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:03.721436 12653 raft_consensus.cc:3037] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:03.723811 12653 raft_consensus.cc:2454] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 1.
I20260812 00:34:03.723819 12356 raft_consensus.cc:2454] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 1.
I20260812 00:34:03.724225 11918 leader_election.cc:302] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b6a059e0a24df3acacc16cbe10733e, 9ae41a64a78145c9af71c25668989429; no voters:
I20260812 00:34:03.724555 12784 raft_consensus.cc:2781] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:03.724748 12784 raft_consensus.cc:686] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [term 1 LEADER]: Becoming Leader. State: Replica: 97b6a059e0a24df3acacc16cbe10733e, State: Running, Role: LEADER
I20260812 00:34:03.725013 12784 consensus_queue.cc:237] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.726686 12793 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: TxnStatusTablet state changed. Reason: New leader 97b6a059e0a24df3acacc16cbe10733e. Latest consensus state: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.727017 12793 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:34:03.727797 12820 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 00:34:03.728072 12820 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 00:34:03.728345 11747 catalog_manager.cc:5495] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65). New cstate: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } health_report { overall_health: UNKNOWN } } }
I20260812 00:34:03.759313 12806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.748367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:39049 (local address 127.0.0.1:39788)
0812 00:34:03.748626 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.748651 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.748817 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.753370 (+ 4553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.753398 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.753411 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:34:03.753591 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.753599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.755053 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.755057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.756046 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.756059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.757123 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.757128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.757283 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.757423 (+ 140us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:34:03.758697 (+ 1274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:34:03.758729 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.758829 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.759012 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 00:34:03.759362 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.748430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.7.244.65:39049)
0812 00:34:03.752723 (+ 4293us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.752730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.752756 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:34:03.752888 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.752894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.753013 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.753272 (+ 259us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:03.753741 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.754924 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.756194 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.756948 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.757111 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.757633 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:03.758526 (+ 893us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:03.758700 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.758888 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.758986 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4137,"thread_start_us":3986,"threads_started":1}
I20260812 00:34:03.768646 12356 raft_consensus.cc:1261] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Refusing update from remote peer 97b6a059e0a24df3acacc16cbe10733e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:03.768646 12653 raft_consensus.cc:1261] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Refusing update from remote peer 97b6a059e0a24df3acacc16cbe10733e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:03.770067 12793 consensus_queue.cc:1035] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 00:34:03.770579 12784 consensus_queue.cc:1035] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:03.780081 12773 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: TxnStatusTablet state changed. Reason: New leader 97b6a059e0a24df3acacc16cbe10733e. Latest consensus state: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.780436 12773 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:03.783432 12793 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.783916 12828 mvcc.cc:205] Tried to move back new op lower bound from 7317482880064000000 to 7317482879900106752. Current Snapshot: MvccSnapshot[applied={T|T < 7317482880064000000}]
I20260812 00:34:03.784035 12793 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:34:03.784327 12824 mvcc.cc:205] Tried to move back new op lower bound from 7317482880064000000 to 7317482879900106752. Current Snapshot: MvccSnapshot[applied={T|T < 7317482880064000000}]
I20260812 00:34:03.788239 12773 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.788518 12773 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 17961f9df85a492db1d4aaa5b20089a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:03.792645 12774 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: TxnStatusTablet state changed. Reason: New leader 97b6a059e0a24df3acacc16cbe10733e. Latest consensus state: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.792928 12774 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:03.795219 12784 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.795564 12784 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 97b6a059e0a24df3acacc16cbe10733e: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:34:03.797609 12829 mvcc.cc:205] Tried to move back new op lower bound from 7317482880064000000 to 7317482879900106752. Current Snapshot: MvccSnapshot[applied={T|T < 7317482880064000000}]
I20260812 00:34:03.799170 12774 tablet_replica.cc:439] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } } }
I20260812 00:34:03.799374 12774 tablet_replica.cc:442] T 9c475a89dd89445ab2b23dbdbe6c3329 P 9ae41a64a78145c9af71c25668989429: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:03.804894 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.796232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.7.244.65:39049)
0812 00:34:03.796830 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.796836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.797841 (+ 1005us) server_negotiation.cc:407] Connection header received
0812 00:34:03.798174 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.798181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.798282 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.798346 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:03.800250 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.801110 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.802815 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.803389 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.803476 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.804007 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:03.804199 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:03.804302 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.804502 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.804577 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452}
I20260812 00:34:03.839352 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.817814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:41617 (local address 127.7.244.65:39049)
0812 00:34:03.818004 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:03.818009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:03.833088 (+ 15079us) server_negotiation.cc:407] Connection header received
0812 00:34:03.833310 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:03.833317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:03.833439 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:03.833555 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:03.834167 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.835374 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.836981 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.837717 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.837859 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.838436 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:03.838582 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:03.838674 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:03.838875 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 00:34:03.838969 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":74}
I20260812 00:34:03.839391 12848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:03.817814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:39049 (local address 127.7.244.65:41617)
0812 00:34:03.832920 (+ 15106us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:03.832943 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:03.833127 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:03.833613 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:03.833617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:03.833628 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:34:03.834003 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.834013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.835555 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.835560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.836822 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:03.836836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:03.837950 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:03.837956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:03.838111 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:03.838186 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:34:03.838707 (+ 521us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:34:03.838735 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 00:34:03.838829 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 00:34:03.838968 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14865,"mutex_wait_us":30,"thread_start_us":15392,"threads_started":1}
I20260812 00:34:03.849689 12653 raft_consensus.cc:1261] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Refusing update from remote peer 97b6a059e0a24df3acacc16cbe10733e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:03.849793 12356 raft_consensus.cc:1261] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Refusing update from remote peer 97b6a059e0a24df3acacc16cbe10733e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:03.850319 12784 consensus_queue.cc:1035] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:03.850622 12793 consensus_queue.cc:1035] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:03.934662 11745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:03.934928 11745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:03.938871 11745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: Sending DeleteTablet for 3 replicas of tablet 28e056e5d796485caf7807e2cf74854a
I20260812 00:34:03.940146 12032 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e056e5d796485caf7807e2cf74854a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:03 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:39774
I20260812 00:34:03.940267 12590 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e056e5d796485caf7807e2cf74854a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:03 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:53742
I20260812 00:34:03.941385 12313 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e056e5d796485caf7807e2cf74854a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:03 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55942
I20260812 00:34:03.941494 11745 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:03.942358 11745 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:03.944083 12859 tablet_replica.cc:330] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: stopping tablet replica
I20260812 00:34:03.944593 12859 raft_consensus.cc:2229] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:03.945333 12859 raft_consensus.cc:2258] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:03.946265 12860 tablet_replica.cc:330] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: stopping tablet replica
I20260812 00:34:03.946762 12860 raft_consensus.cc:2229] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:03.947194 12860 raft_consensus.cc:2258] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:03.950623 12860 ts_tablet_manager.cc:1905] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:03.951978 12858 tablet_replica.cc:330] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: stopping tablet replica
I20260812 00:34:03.952584 12858 raft_consensus.cc:2229] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:03.953059 12858 raft_consensus.cc:2258] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:03.953097 12034 tablet_service.cc:1467] Processing CreateTablet for tablet 08cc5f362d524fabaf55100e496b8d63 (DEFAULT_TABLE table=test-table [id=24bb3bf8e6aa4bc9ae3ba28e5bc504eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:03.953655 12034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08cc5f362d524fabaf55100e496b8d63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.954488 12313 tablet_service.cc:1467] Processing CreateTablet for tablet 08cc5f362d524fabaf55100e496b8d63 (DEFAULT_TABLE table=test-table [id=24bb3bf8e6aa4bc9ae3ba28e5bc504eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:03.954967 12313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08cc5f362d524fabaf55100e496b8d63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.960256 12858 ts_tablet_manager.cc:1905] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:03.961133 12770 tablet_bootstrap.cc:492] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: Bootstrap starting.
I20260812 00:34:03.963040 12770 tablet_bootstrap.cc:654] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.967648 12590 tablet_service.cc:1467] Processing CreateTablet for tablet 08cc5f362d524fabaf55100e496b8d63 (DEFAULT_TABLE table=test-table [id=24bb3bf8e6aa4bc9ae3ba28e5bc504eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:03.968474 12859 ts_tablet_manager.cc:1905] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:03.969027 12590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08cc5f362d524fabaf55100e496b8d63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:03.970866 12860 ts_tablet_manager.cc:1918] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:03.971103 12860 log.cc:1196] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/28e056e5d796485caf7807e2cf74854a
I20260812 00:34:03.971745 12860 ts_tablet_manager.cc:1939] T 28e056e5d796485caf7807e2cf74854a P 17961f9df85a492db1d4aaa5b20089a3: Deleting consensus metadata
I20260812 00:34:03.968142 12858 ts_tablet_manager.cc:1918] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:03.974012 12858 log.cc:1196] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/28e056e5d796485caf7807e2cf74854a
I20260812 00:34:03.974244 11726 catalog_manager.cc:4841] TS 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591): tablet 28e056e5d796485caf7807e2cf74854a (table test-table [id=7517dcb335ac4ef7850ccf702ddcce2d]) successfully deleted
I20260812 00:34:03.974710 12858 ts_tablet_manager.cc:1939] T 28e056e5d796485caf7807e2cf74854a P 9ae41a64a78145c9af71c25668989429: Deleting consensus metadata
I20260812 00:34:03.976822 11726 catalog_manager.cc:4841] TS 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211): tablet 28e056e5d796485caf7807e2cf74854a (table test-table [id=7517dcb335ac4ef7850ccf702ddcce2d]) successfully deleted
I20260812 00:34:03.977283 12766 tablet_bootstrap.cc:492] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: Bootstrap starting.
I20260812 00:34:03.979151 12766 tablet_bootstrap.cc:654] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.980232 12770 tablet_bootstrap.cc:492] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: No bootstrap required, opened a new log
I20260812 00:34:03.980521 12770 ts_tablet_manager.cc:1397] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.000s
I20260812 00:34:03.980598 12768 tablet_bootstrap.cc:492] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: Bootstrap starting.
I20260812 00:34:03.981603 12770 raft_consensus.cc:348] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.982009 12770 raft_consensus.cc:374] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.982149 12770 raft_consensus.cc:729] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b6a059e0a24df3acacc16cbe10733e, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.982595 12770 consensus_queue.cc:260] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.983263 12768 tablet_bootstrap.cc:654] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:03.984795 12770 ts_tablet_manager.cc:1428] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 00:34:03.985028 12859 ts_tablet_manager.cc:1918] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:03.985220 12859 log.cc:1196] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/28e056e5d796485caf7807e2cf74854a
I20260812 00:34:03.985968 12859 ts_tablet_manager.cc:1939] T 28e056e5d796485caf7807e2cf74854a P 97b6a059e0a24df3acacc16cbe10733e: Deleting consensus metadata
I20260812 00:34:03.988111 12766 tablet_bootstrap.cc:492] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: No bootstrap required, opened a new log
I20260812 00:34:03.988343 12766 ts_tablet_manager.cc:1397] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260812 00:34:03.988502 11725 catalog_manager.cc:4841] TS 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049): tablet 28e056e5d796485caf7807e2cf74854a (table test-table [id=7517dcb335ac4ef7850ccf702ddcce2d]) successfully deleted
I20260812 00:34:03.989630 12768 tablet_bootstrap.cc:492] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: No bootstrap required, opened a new log
I20260812 00:34:03.989830 12768 ts_tablet_manager.cc:1397] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260812 00:34:03.989667 12766 raft_consensus.cc:348] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.989965 12766 raft_consensus.cc:374] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.990058 12766 raft_consensus.cc:729] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17961f9df85a492db1d4aaa5b20089a3, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.990325 12766 consensus_queue.cc:260] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.990682 12768 raft_consensus.cc:348] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.990932 12768 raft_consensus.cc:374] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:03.990990 12766 ts_tablet_manager.cc:1428] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 00:34:03.991011 12768 raft_consensus.cc:729] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae41a64a78145c9af71c25668989429, State: Initialized, Role: FOLLOWER
I20260812 00:34:03.991259 12768 consensus_queue.cc:260] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:03.991853 12768 ts_tablet_manager.cc:1428] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:34:04.176429 12793 raft_consensus.cc:482] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:04.176679 12793 raft_consensus.cc:504] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:04.177435 12793 leader_election.cc:288] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591), 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211)
I20260812 00:34:04.178109 12355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08cc5f362d524fabaf55100e496b8d63" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3" is_pre_election: true
I20260812 00:34:04.178185 12653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08cc5f362d524fabaf55100e496b8d63" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae41a64a78145c9af71c25668989429" is_pre_election: true
I20260812 00:34:04.178386 12355 raft_consensus.cc:2454] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 0.
I20260812 00:34:04.178480 12653 raft_consensus.cc:2454] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 0.
I20260812 00:34:04.178951 11918 leader_election.cc:302] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b6a059e0a24df3acacc16cbe10733e, 9ae41a64a78145c9af71c25668989429; no voters:
I20260812 00:34:04.179265 12793 raft_consensus.cc:2781] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:04.179395 12793 raft_consensus.cc:482] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:04.179486 12793 raft_consensus.cc:3037] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:04.181828 12793 raft_consensus.cc:504] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:04.182478 12793 leader_election.cc:288] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 election: Requested vote from peers 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591), 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211)
I20260812 00:34:04.183146 12355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08cc5f362d524fabaf55100e496b8d63" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3"
I20260812 00:34:04.183338 12355 raft_consensus.cc:3037] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:04.183249 12653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08cc5f362d524fabaf55100e496b8d63" candidate_uuid: "97b6a059e0a24df3acacc16cbe10733e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ae41a64a78145c9af71c25668989429"
I20260812 00:34:04.183478 12653 raft_consensus.cc:3037] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:04.185621 12355 raft_consensus.cc:2454] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 1.
I20260812 00:34:04.186122 11918 leader_election.cc:302] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17961f9df85a492db1d4aaa5b20089a3, 97b6a059e0a24df3acacc16cbe10733e; no voters:
I20260812 00:34:04.186465 12793 raft_consensus.cc:2781] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:04.186578 12653 raft_consensus.cc:2454] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b6a059e0a24df3acacc16cbe10733e in term 1.
I20260812 00:34:04.186635 12793 raft_consensus.cc:686] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 1 LEADER]: Becoming Leader. State: Replica: 97b6a059e0a24df3acacc16cbe10733e, State: Running, Role: LEADER
I20260812 00:34:04.186936 12793 consensus_queue.cc:237] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:04.190150 11747 catalog_manager.cc:5495] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e reported cstate change: term changed from 0 to 1, leader changed from <none> to 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65). New cstate: current_term: 1 leader_uuid: "97b6a059e0a24df3acacc16cbe10733e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } health_report { overall_health: UNKNOWN } } }
I20260812 00:34:04.225335 12653 raft_consensus.cc:1261] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Refusing update from remote peer 97b6a059e0a24df3acacc16cbe10733e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:04.226048 12793 consensus_queue.cc:1035] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:04.226266 12356 raft_consensus.cc:1261] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Refusing update from remote peer 97b6a059e0a24df3acacc16cbe10733e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:04.226835 12855 consensus_queue.cc:1035] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 00:34:04.252534 12824 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 00:34:05.854439 11747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:05.854679 11747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:05.859333 11747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: Sending DeleteTablet for 3 replicas of tablet 08cc5f362d524fabaf55100e496b8d63
I20260812 00:34:05.860342 12034 tablet_service.cc:1514] Processing DeleteTablet for tablet 08cc5f362d524fabaf55100e496b8d63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:05 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:39774
I20260812 00:34:05.860589 12313 tablet_service.cc:1514] Processing DeleteTablet for tablet 08cc5f362d524fabaf55100e496b8d63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:05 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55942
I20260812 00:34:05.860602 12590 tablet_service.cc:1514] Processing DeleteTablet for tablet 08cc5f362d524fabaf55100e496b8d63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:05 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:53742
I20260812 00:34:05.873281 12904 tablet_replica.cc:330] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: stopping tablet replica
I20260812 00:34:05.873503 12907 tablet_replica.cc:330] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: stopping tablet replica
I20260812 00:34:05.873790 12904 raft_consensus.cc:2229] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:05.873881 12907 raft_consensus.cc:2229] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:05.874279 12907 raft_consensus.cc:2258] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:05.874347 12904 raft_consensus.cc:2258] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:05.874998 12906 tablet_replica.cc:330] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: stopping tablet replica
I20260812 00:34:05.875290 12906 raft_consensus.cc:2229] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:05.875653 12906 raft_consensus.cc:2258] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:05.875767 12907 ts_tablet_manager.cc:1905] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:05.875885 12904 ts_tablet_manager.cc:1905] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:05.876861 12906 ts_tablet_manager.cc:1905] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:05.883252 12907 ts_tablet_manager.cc:1918] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:34:05.883373 12904 ts_tablet_manager.cc:1918] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:34:05.883555 12907 log.cc:1196] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/08cc5f362d524fabaf55100e496b8d63
I20260812 00:34:05.883622 12904 log.cc:1196] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/08cc5f362d524fabaf55100e496b8d63
I20260812 00:34:05.884142 12904 ts_tablet_manager.cc:1939] T 08cc5f362d524fabaf55100e496b8d63 P 97b6a059e0a24df3acacc16cbe10733e: Deleting consensus metadata
I20260812 00:34:05.884928 12907 ts_tablet_manager.cc:1939] T 08cc5f362d524fabaf55100e496b8d63 P 17961f9df85a492db1d4aaa5b20089a3: Deleting consensus metadata
I20260812 00:34:05.885866 11725 catalog_manager.cc:4841] TS 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049): tablet 08cc5f362d524fabaf55100e496b8d63 (table test-table [id=24bb3bf8e6aa4bc9ae3ba28e5bc504eb]) successfully deleted
I20260812 00:34:05.886282 11726 catalog_manager.cc:4841] TS 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591): tablet 08cc5f362d524fabaf55100e496b8d63 (table test-table [id=24bb3bf8e6aa4bc9ae3ba28e5bc504eb]) successfully deleted
I20260812 00:34:05.888965 12906 ts_tablet_manager.cc:1918] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:34:05.889120 12906 log.cc:1196] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/08cc5f362d524fabaf55100e496b8d63
I20260812 00:34:05.889590 12906 ts_tablet_manager.cc:1939] T 08cc5f362d524fabaf55100e496b8d63 P 9ae41a64a78145c9af71c25668989429: Deleting consensus metadata
I20260812 00:34:05.890836 11726 catalog_manager.cc:4841] TS 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211): tablet 08cc5f362d524fabaf55100e496b8d63 (table test-table [id=24bb3bf8e6aa4bc9ae3ba28e5bc504eb]) successfully deleted
Aug 12 00:34:05 dist-test-slave-gjw7 krb5kdc[11669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:34:05 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:05.932020 12915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:05.916354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42706 (local address 127.7.244.126:44065)
0812 00:34:05.916985 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:05.916992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:05.917281 (+ 289us) server_negotiation.cc:407] Connection header received
0812 00:34:05.917529 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:05.917533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:05.917605 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:05.917698 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:05.919146 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:05.919940 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:05.921366 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:05.921794 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:05.921959 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:05.926125 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:05.926132 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:05.926136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:05.926181 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:05.929014 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:05.929869 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:05.929882 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:05.929884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:05.930039 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:05.930447 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:05.930451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:05.930453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:05.930777 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:05.931067 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:05.931383 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 00:34:05.931644 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":285,"threads_started":1}
I20260812 00:34:05.936005 11747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:05.936666 11747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:05.943740 12034 tablet_service.cc:1467] Processing CreateTablet for tablet f94d6e8f5efa4b19acb0d50c49c39d7e (DEFAULT_TABLE table=test-table [id=944ad761c09846d2b80b99efb649d7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:05.944106 12590 tablet_service.cc:1467] Processing CreateTablet for tablet f94d6e8f5efa4b19acb0d50c49c39d7e (DEFAULT_TABLE table=test-table [id=944ad761c09846d2b80b99efb649d7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:05.944115 12313 tablet_service.cc:1467] Processing CreateTablet for tablet f94d6e8f5efa4b19acb0d50c49c39d7e (DEFAULT_TABLE table=test-table [id=944ad761c09846d2b80b99efb649d7a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:05.944293 12034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f94d6e8f5efa4b19acb0d50c49c39d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:05.944571 12590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f94d6e8f5efa4b19acb0d50c49c39d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:05.944669 12313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f94d6e8f5efa4b19acb0d50c49c39d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:05.951718 12918 tablet_bootstrap.cc:492] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: Bootstrap starting.
I20260812 00:34:05.953531 12918 tablet_bootstrap.cc:654] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:05.958036 12923 tablet_bootstrap.cc:492] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: Bootstrap starting.
I20260812 00:34:05.958839 12919 tablet_bootstrap.cc:492] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: Bootstrap starting.
I20260812 00:34:05.961138 12919 tablet_bootstrap.cc:654] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:05.964430 12923 tablet_bootstrap.cc:654] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:05.965544 12919 tablet_bootstrap.cc:492] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: No bootstrap required, opened a new log
I20260812 00:34:05.965714 12918 tablet_bootstrap.cc:492] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: No bootstrap required, opened a new log
I20260812 00:34:05.965744 12919 ts_tablet_manager.cc:1397] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.003s
I20260812 00:34:05.965939 12918 ts_tablet_manager.cc:1397] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260812 00:34:05.966535 12919 raft_consensus.cc:348] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:05.966809 12919 raft_consensus.cc:374] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:05.966763 12918 raft_consensus.cc:348] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:05.966976 12919 raft_consensus.cc:729] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae41a64a78145c9af71c25668989429, State: Initialized, Role: FOLLOWER
I20260812 00:34:05.967020 12918 raft_consensus.cc:374] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:05.967116 12918 raft_consensus.cc:729] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17961f9df85a492db1d4aaa5b20089a3, State: Initialized, Role: FOLLOWER
I20260812 00:34:05.967347 12923 tablet_bootstrap.cc:492] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: No bootstrap required, opened a new log
I20260812 00:34:05.967343 12919 consensus_queue.cc:260] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:05.967370 12918 consensus_queue.cc:260] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:05.967514 12923 ts_tablet_manager.cc:1397] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260812 00:34:05.968341 12919 ts_tablet_manager.cc:1428] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:34:05.968777 12923 raft_consensus.cc:348] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:05.969079 12923 raft_consensus.cc:374] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:05.969194 12923 raft_consensus.cc:729] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b6a059e0a24df3acacc16cbe10733e, State: Initialized, Role: FOLLOWER
I20260812 00:34:05.969470 12918 ts_tablet_manager.cc:1428] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 00:34:05.969537 12923 consensus_queue.cc:260] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:05.970151 12923 ts_tablet_manager.cc:1428] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:34:06.151032 12935 raft_consensus.cc:482] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:06.151453 12935 raft_consensus.cc:504] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.153496 12935 leader_election.cc:288] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049), 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591)
I20260812 00:34:06.167774 12944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.154477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:48885 (local address 127.7.244.65:39049)
0812 00:34:06.158937 (+ 4460us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:06.158943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:06.159394 (+ 451us) server_negotiation.cc:407] Connection header received
0812 00:34:06.159491 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:06.159496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:06.159573 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:06.159671 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:06.160484 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.161845 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.164996 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.166393 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.166609 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.166696 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:06.167176 (+ 480us) server_negotiation.cc:299] Negotiation successful
0812 00:34:06.167338 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4311,"thread_start_us":4121,"threads_started":1}
I20260812 00:34:06.167747 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.154121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:52723 (local address 127.7.244.66:33591)
0812 00:34:06.158393 (+ 4272us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:06.158400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:06.158819 (+ 419us) server_negotiation.cc:407] Connection header received
0812 00:34:06.159266 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:06.159272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:06.159473 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:06.159595 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:06.160456 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.161845 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.164993 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.166362 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.166609 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.166696 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:06.167175 (+ 479us) server_negotiation.cc:299] Negotiation successful
0812 00:34:06.167338 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4085,"thread_start_us":3905,"threads_started":1}
I20260812 00:34:06.167881 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.153812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:33591 (local address 127.7.244.67:52723)
0812 00:34:06.158657 (+ 4845us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:06.158683 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:06.159057 (+ 374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:06.159642 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:06.159646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:06.159679 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:06.160159 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:06.160177 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.162053 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.162060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:06.164719 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:06.164744 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.166701 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.166708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:06.166912 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.167011 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 00:34:06.167129 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 00:34:06.167329 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4697,"mutex_wait_us":131,"thread_start_us":4506,"threads_started":1}
I20260812 00:34:06.167928 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.153827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:39049 (local address 127.7.244.67:48885)
0812 00:34:06.159167 (+ 5340us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:06.159184 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:06.159311 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:06.159702 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:06.159706 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:06.159772 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:06.160190 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:06.160200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.162053 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.162060 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:06.164718 (+ 2658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:06.164744 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.166667 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.166679 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:06.166974 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.167024 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 00:34:06.167129 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 00:34:06.167353 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5206,"mutex_wait_us":261,"spinlock_wait_cycles":3968,"thread_start_us":4900,"threads_started":1}
I20260812 00:34:06.168804 12355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f94d6e8f5efa4b19acb0d50c49c39d7e" candidate_uuid: "9ae41a64a78145c9af71c25668989429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3" is_pre_election: true
I20260812 00:34:06.169081 12355 raft_consensus.cc:2454] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae41a64a78145c9af71c25668989429 in term 0.
I20260812 00:34:06.169186 12077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f94d6e8f5efa4b19acb0d50c49c39d7e" candidate_uuid: "9ae41a64a78145c9af71c25668989429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b6a059e0a24df3acacc16cbe10733e" is_pre_election: true
I20260812 00:34:06.169699 12077 raft_consensus.cc:2454] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae41a64a78145c9af71c25668989429 in term 0.
I20260812 00:34:06.169746 12491 leader_election.cc:302] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17961f9df85a492db1d4aaa5b20089a3, 9ae41a64a78145c9af71c25668989429; no voters:
I20260812 00:34:06.170267 12935 raft_consensus.cc:2781] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:06.170393 12935 raft_consensus.cc:482] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:06.170506 12935 raft_consensus.cc:3037] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:06.172914 12935 raft_consensus.cc:504] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.173586 12935 leader_election.cc:288] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [CANDIDATE]: Term 1 election: Requested vote from peers 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049), 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591)
I20260812 00:34:06.174198 12355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f94d6e8f5efa4b19acb0d50c49c39d7e" candidate_uuid: "9ae41a64a78145c9af71c25668989429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3"
I20260812 00:34:06.174216 12077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f94d6e8f5efa4b19acb0d50c49c39d7e" candidate_uuid: "9ae41a64a78145c9af71c25668989429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b6a059e0a24df3acacc16cbe10733e"
I20260812 00:34:06.174423 12355 raft_consensus.cc:3037] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:06.174414 12077 raft_consensus.cc:3037] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:06.176712 12355 raft_consensus.cc:2454] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae41a64a78145c9af71c25668989429 in term 1.
I20260812 00:34:06.176831 12077 raft_consensus.cc:2454] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae41a64a78145c9af71c25668989429 in term 1.
I20260812 00:34:06.177258 12491 leader_election.cc:302] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17961f9df85a492db1d4aaa5b20089a3, 9ae41a64a78145c9af71c25668989429; no voters:
I20260812 00:34:06.177634 12935 raft_consensus.cc:2781] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:06.178129 12935 raft_consensus.cc:686] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 1 LEADER]: Becoming Leader. State: Replica: 9ae41a64a78145c9af71c25668989429, State: Running, Role: LEADER
I20260812 00:34:06.178614 12935 consensus_queue.cc:237] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.182260 11747 catalog_manager.cc:5495] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ae41a64a78145c9af71c25668989429 (127.7.244.67). New cstate: current_term: 1 leader_uuid: "9ae41a64a78145c9af71c25668989429" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } health_report { overall_health: HEALTHY } } }
I20260812 00:34:06.209925 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.203551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53744 (local address 127.7.244.67:42211)
0812 00:34:06.204154 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:06.204159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:06.204180 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:34:06.204285 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:06.204288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:06.204378 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:06.204530 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:06.204948 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.205817 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.207331 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.207787 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.207902 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.208490 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:06.209144 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:06.209368 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:06.209472 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 00:34:06.209554 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":335,"threads_started":1}
I20260812 00:34:06.227530 12944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.221650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:57731 (local address 127.7.244.65:39049)
0812 00:34:06.221840 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:06.221845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:06.222388 (+ 543us) server_negotiation.cc:407] Connection header received
0812 00:34:06.222556 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:06.222560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:06.222633 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:06.222743 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:06.223501 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.224225 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.225779 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.226208 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.226353 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.226900 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:06.227036 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:06.227110 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:06.227243 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 00:34:06.227321 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:34:06.227633 12955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.221511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:39049 (local address 127.7.244.67:57731)
0812 00:34:06.222270 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:06.222287 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:06.222441 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:06.222798 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:06.222804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:06.222820 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:34:06.223233 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:06.223246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.224392 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.224396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:06.225603 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:06.225621 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.226480 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.226486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:06.226626 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.226700 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:34:06.227119 (+ 419us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:34:06.227127 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:34:06.227184 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 00:34:06.227309 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":401,"threads_started":1}
I20260812 00:34:06.237090 12077 raft_consensus.cc:1261] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Refusing update from remote peer 9ae41a64a78145c9af71c25668989429: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:06.237886 12355 raft_consensus.cc:1261] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Refusing update from remote peer 9ae41a64a78145c9af71c25668989429: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:06.238057 12945 consensus_queue.cc:1035] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:06.238523 12935 consensus_queue.cc:1035] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:06.282542 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.267268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:49237 (local address 127.7.244.67:42211)
0812 00:34:06.267486 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:06.267494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:06.277550 (+ 10056us) server_negotiation.cc:407] Connection header received
0812 00:34:06.277751 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:06.277756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:06.277851 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:06.277945 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:06.278470 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.279272 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.280681 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.281098 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.281213 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.281766 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:06.281887 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:06.281966 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:06.282120 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 00:34:06.282207 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:34:06.282614 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:06.266628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:42211 (local address 127.7.244.65:49237)
0812 00:34:06.277427 (+ 10799us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:06.277446 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:06.277603 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:06.277992 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:06.277996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:06.278010 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:34:06.278300 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:06.278310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.279429 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.279435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:06.280521 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:06.280537 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:06.281317 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:06.281326 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:06.281532 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:06.281593 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:34:06.281993 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:34:06.282008 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 00:34:06.282086 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 00:34:06.282240 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10592,"thread_start_us":11067,"threads_started":1}
I20260812 00:34:06.321861 11743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:06.322140 11743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:06.325799 11743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: Sending DeleteTablet for 3 replicas of tablet f94d6e8f5efa4b19acb0d50c49c39d7e
I20260812 00:34:06.326857 12313 tablet_service.cc:1514] Processing DeleteTablet for tablet f94d6e8f5efa4b19acb0d50c49c39d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:06 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55942
I20260812 00:34:06.326913 12590 tablet_service.cc:1514] Processing DeleteTablet for tablet f94d6e8f5efa4b19acb0d50c49c39d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:06 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:53742
I20260812 00:34:06.327307 12906 tablet_replica.cc:330] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: stopping tablet replica
I20260812 00:34:06.327539 12906 raft_consensus.cc:2229] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:06.327679 12907 tablet_replica.cc:330] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: stopping tablet replica
I20260812 00:34:06.327922 12907 raft_consensus.cc:2229] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:06.328156 12907 raft_consensus.cc:2258] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:06.328159 12906 raft_consensus.cc:2258] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:06.328301 11743 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:06.329366 11743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:06.329577 12907 ts_tablet_manager.cc:1905] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:06.329578 12906 ts_tablet_manager.cc:1905] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:06.336773 12906 ts_tablet_manager.cc:1918] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:06.337217 12906 log.cc:1196] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/f94d6e8f5efa4b19acb0d50c49c39d7e
I20260812 00:34:06.337523 12034 tablet_service.cc:1514] Processing DeleteTablet for tablet f94d6e8f5efa4b19acb0d50c49c39d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:06 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:39774
I20260812 00:34:06.337867 12904 tablet_replica.cc:330] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: stopping tablet replica
I20260812 00:34:06.337886 12906 ts_tablet_manager.cc:1939] T f94d6e8f5efa4b19acb0d50c49c39d7e P 9ae41a64a78145c9af71c25668989429: Deleting consensus metadata
I20260812 00:34:06.338100 12904 raft_consensus.cc:2229] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:06.338327 12904 raft_consensus.cc:2258] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:06.339231 11726 catalog_manager.cc:4841] TS 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211): tablet f94d6e8f5efa4b19acb0d50c49c39d7e (table test-table [id=944ad761c09846d2b80b99efb649d7a1]) successfully deleted
I20260812 00:34:06.342445 12907 ts_tablet_manager.cc:1918] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:06.342806 12907 log.cc:1196] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/f94d6e8f5efa4b19acb0d50c49c39d7e
I20260812 00:34:06.343425 12907 ts_tablet_manager.cc:1939] T f94d6e8f5efa4b19acb0d50c49c39d7e P 17961f9df85a492db1d4aaa5b20089a3: Deleting consensus metadata
I20260812 00:34:06.343724 12904 ts_tablet_manager.cc:1905] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:06.344581 12313 tablet_service.cc:1467] Processing CreateTablet for tablet b178ab5ba34641a4adcdc2ea9015de72 (DEFAULT_TABLE table=test-table [id=35693477e5644e71bcb1f8f3d9e8bf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:06.345082 12313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b178ab5ba34641a4adcdc2ea9015de72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:06.345950 12590 tablet_service.cc:1467] Processing CreateTablet for tablet b178ab5ba34641a4adcdc2ea9015de72 (DEFAULT_TABLE table=test-table [id=35693477e5644e71bcb1f8f3d9e8bf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:06.346441 12590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b178ab5ba34641a4adcdc2ea9015de72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:06.346570 12034 tablet_service.cc:1467] Processing CreateTablet for tablet b178ab5ba34641a4adcdc2ea9015de72 (DEFAULT_TABLE table=test-table [id=35693477e5644e71bcb1f8f3d9e8bf52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:06.347141 12034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b178ab5ba34641a4adcdc2ea9015de72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:06.352214 11726 catalog_manager.cc:4841] TS 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591): tablet f94d6e8f5efa4b19acb0d50c49c39d7e (table test-table [id=944ad761c09846d2b80b99efb649d7a1]) successfully deleted
I20260812 00:34:06.353260 12918 tablet_bootstrap.cc:492] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: Bootstrap starting.
I20260812 00:34:06.355059 12919 tablet_bootstrap.cc:492] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: Bootstrap starting.
I20260812 00:34:06.355093 12918 tablet_bootstrap.cc:654] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:06.357050 12919 tablet_bootstrap.cc:654] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:06.364322 12919 tablet_bootstrap.cc:492] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: No bootstrap required, opened a new log
I20260812 00:34:06.364636 12919 ts_tablet_manager.cc:1397] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260812 00:34:06.365775 12919 raft_consensus.cc:348] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.366195 12919 raft_consensus.cc:374] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:06.366312 12919 raft_consensus.cc:729] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae41a64a78145c9af71c25668989429, State: Initialized, Role: FOLLOWER
I20260812 00:34:06.366684 12919 consensus_queue.cc:260] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.367425 12919 ts_tablet_manager.cc:1428] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:34:06.367619 12904 ts_tablet_manager.cc:1918] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:06.367810 12904 log.cc:1196] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/f94d6e8f5efa4b19acb0d50c49c39d7e
I20260812 00:34:06.377040 12904 ts_tablet_manager.cc:1939] T f94d6e8f5efa4b19acb0d50c49c39d7e P 97b6a059e0a24df3acacc16cbe10733e: Deleting consensus metadata
I20260812 00:34:06.377542 12923 tablet_bootstrap.cc:492] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: Bootstrap starting.
I20260812 00:34:06.379462 12923 tablet_bootstrap.cc:654] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:06.387135 11725 catalog_manager.cc:4841] TS 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049): tablet f94d6e8f5efa4b19acb0d50c49c39d7e (table test-table [id=944ad761c09846d2b80b99efb649d7a1]) successfully deleted
I20260812 00:34:06.390712 12923 tablet_bootstrap.cc:492] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: No bootstrap required, opened a new log
I20260812 00:34:06.390939 12923 ts_tablet_manager.cc:1397] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.005s
I20260812 00:34:06.391768 12918 tablet_bootstrap.cc:492] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: No bootstrap required, opened a new log
I20260812 00:34:06.391950 12918 ts_tablet_manager.cc:1397] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.023s
I20260812 00:34:06.391831 12923 raft_consensus.cc:348] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.392120 12923 raft_consensus.cc:374] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:06.392215 12923 raft_consensus.cc:729] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b6a059e0a24df3acacc16cbe10733e, State: Initialized, Role: FOLLOWER
I20260812 00:34:06.392496 12923 consensus_queue.cc:260] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.392680 12918 raft_consensus.cc:348] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.392966 12918 raft_consensus.cc:374] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:06.393047 12918 raft_consensus.cc:729] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17961f9df85a492db1d4aaa5b20089a3, State: Initialized, Role: FOLLOWER
I20260812 00:34:06.393187 12923 ts_tablet_manager.cc:1428] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260812 00:34:06.393383 12918 consensus_queue.cc:260] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.393945 12918 ts_tablet_manager.cc:1428] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 00:34:06.689903 12935 raft_consensus.cc:482] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:06.690168 12935 raft_consensus.cc:504] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.691123 12935 leader_election.cc:288] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049), 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591)
I20260812 00:34:06.691794 12077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b178ab5ba34641a4adcdc2ea9015de72" candidate_uuid: "9ae41a64a78145c9af71c25668989429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b6a059e0a24df3acacc16cbe10733e" is_pre_election: true
I20260812 00:34:06.691872 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b178ab5ba34641a4adcdc2ea9015de72" candidate_uuid: "9ae41a64a78145c9af71c25668989429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3" is_pre_election: true
I20260812 00:34:06.692101 12077 raft_consensus.cc:2454] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae41a64a78145c9af71c25668989429 in term 0.
I20260812 00:34:06.692121 12356 raft_consensus.cc:2454] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ae41a64a78145c9af71c25668989429 in term 0.
I20260812 00:34:06.692678 12491 leader_election.cc:302] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 17961f9df85a492db1d4aaa5b20089a3, 97b6a059e0a24df3acacc16cbe10733e, 9ae41a64a78145c9af71c25668989429; no voters:
I20260812 00:34:06.693015 12935 raft_consensus.cc:2781] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:06.693171 12935 raft_consensus.cc:482] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:06.693318 12935 raft_consensus.cc:3037] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:06.697031 12935 raft_consensus.cc:504] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.697934 12935 leader_election.cc:288] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [CANDIDATE]: Term 1 election: Requested vote from peers 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049), 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591)
I20260812 00:34:06.698611 12077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b178ab5ba34641a4adcdc2ea9015de72" candidate_uuid: "9ae41a64a78145c9af71c25668989429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b6a059e0a24df3acacc16cbe10733e"
I20260812 00:34:06.698642 12356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b178ab5ba34641a4adcdc2ea9015de72" candidate_uuid: "9ae41a64a78145c9af71c25668989429" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17961f9df85a492db1d4aaa5b20089a3"
I20260812 00:34:06.698850 12077 raft_consensus.cc:3037] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:06.698921 12356 raft_consensus.cc:3037] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:06.701013 12077 raft_consensus.cc:2454] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae41a64a78145c9af71c25668989429 in term 1.
I20260812 00:34:06.701592 12489 leader_election.cc:302] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b6a059e0a24df3acacc16cbe10733e, 9ae41a64a78145c9af71c25668989429; no voters:
I20260812 00:34:06.701974 12935 raft_consensus.cc:2781] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:06.702096 12356 raft_consensus.cc:2454] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ae41a64a78145c9af71c25668989429 in term 1.
I20260812 00:34:06.702173 12935 raft_consensus.cc:686] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 1 LEADER]: Becoming Leader. State: Replica: 9ae41a64a78145c9af71c25668989429, State: Running, Role: LEADER
I20260812 00:34:06.702477 12935 consensus_queue.cc:237] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } }
I20260812 00:34:06.705868 11743 catalog_manager.cc:5495] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ae41a64a78145c9af71c25668989429 (127.7.244.67). New cstate: current_term: 1 leader_uuid: "9ae41a64a78145c9af71c25668989429" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ae41a64a78145c9af71c25668989429" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 42211 } health_report { overall_health: HEALTHY } } }
I20260812 00:34:06.758550 12356 raft_consensus.cc:1261] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Refusing update from remote peer 9ae41a64a78145c9af71c25668989429: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:06.758584 12077 raft_consensus.cc:1261] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Refusing update from remote peer 9ae41a64a78145c9af71c25668989429: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:06.759122 12945 consensus_queue.cc:1035] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17961f9df85a492db1d4aaa5b20089a3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:06.759443 12935 consensus_queue.cc:1035] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b6a059e0a24df3acacc16cbe10733e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 39049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 00:34:06.782819 12879 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 00:34:08.384562 11745 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:08.384833 11745 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:08.389456 11745 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98eeb76b992e451f8842ea70656414e6: Sending DeleteTablet for 3 replicas of tablet b178ab5ba34641a4adcdc2ea9015de72
I20260812 00:34:08.390470 12034 tablet_service.cc:1514] Processing DeleteTablet for tablet b178ab5ba34641a4adcdc2ea9015de72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:08 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:39774
I20260812 00:34:08.390503 12313 tablet_service.cc:1514] Processing DeleteTablet for tablet b178ab5ba34641a4adcdc2ea9015de72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:08 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55942
I20260812 00:34:08.390523 12590 tablet_service.cc:1514] Processing DeleteTablet for tablet b178ab5ba34641a4adcdc2ea9015de72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:08 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:53742
I20260812 00:34:08.394171 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11886
I20260812 00:34:08.407188 13029 tablet_replica.cc:330] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: stopping tablet replica
I20260812 00:34:08.407965 13031 tablet_replica.cc:330] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: stopping tablet replica
I20260812 00:34:08.408353 13031 raft_consensus.cc:2229] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:08.408474 13028 tablet_replica.cc:330] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: stopping tablet replica
I20260812 00:34:08.408653 13029 raft_consensus.cc:2229] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:08.408702 13028 raft_consensus.cc:2229] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:08.408854 13031 raft_consensus.cc:2258] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:08.409030 13029 raft_consensus.cc:2258] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:08.409137 13028 raft_consensus.cc:2258] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:08.410187 13029 ts_tablet_manager.cc:1905] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:08.410187 13031 ts_tablet_manager.cc:1905] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:08.410745 13028 ts_tablet_manager.cc:1905] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:08.417766 13031 ts_tablet_manager.cc:1918] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:34:08.417766 13029 ts_tablet_manager.cc:1918] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:34:08.417961 13029 log.cc:1196] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/b178ab5ba34641a4adcdc2ea9015de72
I20260812 00:34:08.418045 13028 ts_tablet_manager.cc:1918] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 00:34:08.418113 13031 log.cc:1196] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/b178ab5ba34641a4adcdc2ea9015de72
I20260812 00:34:08.418220 13028 log.cc:1196] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/b178ab5ba34641a4adcdc2ea9015de72
I20260812 00:34:08.418568 13029 ts_tablet_manager.cc:1939] T b178ab5ba34641a4adcdc2ea9015de72 P 97b6a059e0a24df3acacc16cbe10733e: Deleting consensus metadata
I20260812 00:34:08.418584 13031 ts_tablet_manager.cc:1939] T b178ab5ba34641a4adcdc2ea9015de72 P 17961f9df85a492db1d4aaa5b20089a3: Deleting consensus metadata
I20260812 00:34:08.418690 13028 ts_tablet_manager.cc:1939] T b178ab5ba34641a4adcdc2ea9015de72 P 9ae41a64a78145c9af71c25668989429: Deleting consensus metadata
I20260812 00:34:08.419822 11726 catalog_manager.cc:4841] TS 17961f9df85a492db1d4aaa5b20089a3 (127.7.244.66:33591): tablet b178ab5ba34641a4adcdc2ea9015de72 (table test-table [id=35693477e5644e71bcb1f8f3d9e8bf52]) successfully deleted
I20260812 00:34:08.419848 11725 catalog_manager.cc:4841] TS 97b6a059e0a24df3acacc16cbe10733e (127.7.244.65:39049): tablet b178ab5ba34641a4adcdc2ea9015de72 (table test-table [id=35693477e5644e71bcb1f8f3d9e8bf52]) successfully deleted
I20260812 00:34:08.420118 11726 catalog_manager.cc:4841] TS 9ae41a64a78145c9af71c25668989429 (127.7.244.67:42211): tablet b178ab5ba34641a4adcdc2ea9015de72 (table test-table [id=35693477e5644e71bcb1f8f3d9e8bf52]) successfully deleted
Aug 12 00:34:08 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:34:08.431392 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:08.395724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.7.244.65:39049)
0812 00:34:08.407564 (+ 11840us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:08.407572 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:08.408372 (+ 800us) server_negotiation.cc:407] Connection header received
0812 00:34:08.408457 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:08.408462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:08.408546 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:08.408673 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:08.411277 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:08.412684 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:08.414760 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:08.416520 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:08.416686 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:08.424943 (+ 8257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:08.424963 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:08.424981 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:08.425026 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:08.428080 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:08.429019 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:08.429028 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:08.429030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:08.429197 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:08.429667 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:08.429671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:08.429674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:08.430091 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:08.430421 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:08.430752 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 00:34:08.431013 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11613,"thread_start_us":11400,"threads_started":1}
W20260812 00:34:08.432169 12166 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39812
W20260812 00:34:08.433007 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11886: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:34:08.433341 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 11886
I20260812 00:34:08.472398 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 12179
Aug 12 00:34:08 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:34:08.488412 13035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:08.473160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55944 (local address 127.7.244.66:33591)
0812 00:34:08.473713 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:08.473717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:08.473734 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:34:08.473795 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:08.473799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:08.473863 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:08.473976 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:08.475357 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:08.476165 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:08.477614 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:08.478056 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:08.478159 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:08.482568 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:08.482589 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:08.482606 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:08.482649 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:08.485555 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:08.486331 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:08.486340 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:08.486342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:08.486501 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:08.486951 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:08.486954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:08.486956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:08.487319 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:08.487565 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:08.487902 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 00:34:08.488097 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":305,"threads_started":1}
W20260812 00:34:08.489110 12429 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55944
W20260812 00:34:08.489820 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 12179: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:34:08.489967 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 12179
I20260812 00:34:08.532668 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 12446
Aug 12 00:34:08 dist-test-slave-gjw7 krb5kdc[11669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494845, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:34:08.550390 13036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:08.533566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53758 (local address 127.7.244.67:42211)
0812 00:34:08.534107 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:08.534112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:08.534130 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:34:08.534186 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:08.534190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:08.534254 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:08.534345 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:08.535806 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:08.536601 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:08.538087 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:08.538554 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:08.538669 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:08.543429 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:08.543452 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:08.543469 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:08.543532 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:08.546970 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:08.547885 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:08.547896 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:08.547899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:08.548072 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:08.548550 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:08.548555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:08.548557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:08.548987 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:08.549242 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:08.549715 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 00:34:08.549958 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":290,"threads_started":1}
W20260812 00:34:08.551225 12718 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53758
W20260812 00:34:08.552034 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 12446: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:34:08.552232 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 12446
I20260812 00:34:08.604619 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11685
I20260812 00:34:08.605873 11832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:08.725363 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 11685
2026-08-12T00:34:08Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9422 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:34:08 dist-test-slave-gjw7 krb5kdc[13042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:34:08 dist-test-slave-gjw7 krb5kdc[13042](info): set up 2 sockets
Aug 12 00:34:08 dist-test-slave-gjw7 krb5kdc[13042](info): commencing operation
krb5kdc: starting...
W20260812 00:34:10.031840 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:10 dist-test-slave-gjw7 krb5kdc[13042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:34:10.263392 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:34705
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:34705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:10.620764 13064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:10.621140 13064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:10.621284 13064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:10.629058 13064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:10.629217 13064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:10.629345 13064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:10.629431 13064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:10.629496 13064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:10.640198 13064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:34705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:34705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:10.641774 13064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:10.643250 13064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:10.656531 13072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:10.656620 13075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:10.656801 13064 server_base.cc:1034] running on GCE node
W20260812 00:34:10.656599 13077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:10.658087 13064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:10.659713 13064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:10.660982 13064 hybrid_clock.cc:648] HybridClock initialized: now 1786494850660918 us; error 47 us; skew 500 ppm
I20260812 00:34:10.661474 13064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:10 dist-test-slave-gjw7 krb5kdc[13042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494850, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:10.665108 13064 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:34:10.666929 13064 webserver.cc:460] Webserver started at http://127.7.244.126:39703/ using document root <none> and password file <none>
I20260812 00:34:10.667723 13064 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:10.667860 13064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:10.668195 13064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:10.671511 13064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "fd9e7e22231047e895339b9dd59a6652"
format_stamp: "Formatted at 2026-08-12 00:34:10 on dist-test-slave-gjw7"
server_key: "2dbcb458fa2f682fb625b202c9e21ec7"
server_key_iv: "6be7126a78a92b545e3b0d35b22e5ce5"
server_key_version: "encryptionkey@0"
I20260812 00:34:10.672412 13064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "fd9e7e22231047e895339b9dd59a6652"
format_stamp: "Formatted at 2026-08-12 00:34:10 on dist-test-slave-gjw7"
server_key: "2dbcb458fa2f682fb625b202c9e21ec7"
server_key_iv: "6be7126a78a92b545e3b0d35b22e5ce5"
server_key_version: "encryptionkey@0"
I20260812 00:34:10.678371 13064 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 00:34:10.682390 13094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:10.683991 13064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 00:34:10.684202 13064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "fd9e7e22231047e895339b9dd59a6652"
format_stamp: "Formatted at 2026-08-12 00:34:10 on dist-test-slave-gjw7"
server_key: "2dbcb458fa2f682fb625b202c9e21ec7"
server_key_iv: "6be7126a78a92b545e3b0d35b22e5ce5"
server_key_version: "encryptionkey@0"
I20260812 00:34:10.684449 13064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:10.720161 13064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:10.724624 13064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:10.724933 13064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:10.744760 13064 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:34705
I20260812 00:34:10.744807 13199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:34705 every 8 connection(s)
I20260812 00:34:10.746381 13064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:10.746915 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 13064
I20260812 00:34:10.747151 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:10.747967 8145 external_mini_cluster.cc:1428] Setting key 07969e72d00542059c0f9828e3c834ed
I20260812 00:34:10.751397 13200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 00:34:10 dist-test-slave-gjw7 krb5kdc[13042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:10.765964 13200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652: Bootstrap starting.
I20260812 00:34:10.770538 13200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:10.771941 13200 log.cc:824] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:10.775712 13203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:10.750253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51452 (local address 127.7.244.126:34705)
0812 00:34:10.751183 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:10.751200 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:10.751255 (+ 55us) server_negotiation.cc:407] Connection header received
0812 00:34:10.753634 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:10.753668 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:10.754210 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:10.754696 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:10.756321 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:10.758594 (+ 2273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:10.760250 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:10.761421 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:10.761641 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:10.765922 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:10.765936 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:10.765965 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:10.766029 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:10.770007 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:10.771020 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:10.771033 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:10.771044 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:10.771293 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:10.771935 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:10.771940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:10.771943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:10.772809 (+ 866us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:10.773099 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:10.773739 (+ 640us) server_negotiation.cc:299] Negotiation successful
0812 00:34:10.774275 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":361,"threads_started":1}
I20260812 00:34:10.776122 13200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652: No bootstrap required, opened a new log
I20260812 00:34:10.785014 13200 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e7e22231047e895339b9dd59a6652" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 34705 } }
I20260812 00:34:10.785530 13200 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:10.785640 13200 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd9e7e22231047e895339b9dd59a6652, State: Initialized, Role: FOLLOWER
I20260812 00:34:10.786258 13200 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e7e22231047e895339b9dd59a6652" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 34705 } }
I20260812 00:34:10.786523 13200 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:10.786669 13200 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:10.786854 13200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:10.789271 13200 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e7e22231047e895339b9dd59a6652" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 34705 } }
I20260812 00:34:10.789909 13200 leader_election.cc:302] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd9e7e22231047e895339b9dd59a6652; no voters:
I20260812 00:34:10.790771 13200 leader_election.cc:288] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:10.790918 13209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:10.791618 13209 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [term 1 LEADER]: Becoming Leader. State: Replica: fd9e7e22231047e895339b9dd59a6652, State: Running, Role: LEADER
I20260812 00:34:10.792335 13209 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e7e22231047e895339b9dd59a6652" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 34705 } }
I20260812 00:34:10.793023 13200 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:10.795533 13215 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd9e7e22231047e895339b9dd59a6652" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e7e22231047e895339b9dd59a6652" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 34705 } } }
I20260812 00:34:10.796061 13215 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:10.795604 13216 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd9e7e22231047e895339b9dd59a6652. Latest consensus state: current_term: 1 leader_uuid: "fd9e7e22231047e895339b9dd59a6652" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e7e22231047e895339b9dd59a6652" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 34705 } } }
I20260812 00:34:10.796303 13216 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:10.798424 13221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:10.804900 13221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:10.814961 13221 catalog_manager.cc:1347] Generated new cluster ID: f30e6e33ef924713a35b1099feed37ed
I20260812 00:34:10.815111 13221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:10.834317 13221 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:10.835893 13221 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:10.848001 13221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd9e7e22231047e895339b9dd59a6652: Generated new TSK 0
I20260812 00:34:10.848955 13221 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:34:10.917770 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:34705
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:11.286623 13246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:11.286972 13246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:11.287114 13246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:11.295004 13246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:11.295156 13246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:11.295315 13246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:11.305950 13246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:11.307365 13246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:11.308763 13246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:11.322953 13254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:11.322888 13253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:11.322930 13258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:11.324295 13246 server_base.cc:1034] running on GCE node
I20260812 00:34:11.325090 13246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:11.326280 13246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:11.327543 13246 hybrid_clock.cc:648] HybridClock initialized: now 1786494851327481 us; error 50 us; skew 500 ppm
I20260812 00:34:11.327934 13246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:11 dist-test-slave-gjw7 krb5kdc[13042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:11.331748 13246 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:34:11.333508 13246 webserver.cc:460] Webserver started at http://127.7.244.65:42757/ using document root <none> and password file <none>
I20260812 00:34:11.334340 13246 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:11.334483 13246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:11.334824 13246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:11.338562 13246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "f1d7b2546608451ba09d6acafaa55c26"
format_stamp: "Formatted at 2026-08-12 00:34:11 on dist-test-slave-gjw7"
server_key: "ef1389a0fc0cc51b05fff0f7d813f65a"
server_key_iv: "1c42f589d6f42eadf72e2ade5521cd17"
server_key_version: "encryptionkey@0"
I20260812 00:34:11.339545 13246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "f1d7b2546608451ba09d6acafaa55c26"
format_stamp: "Formatted at 2026-08-12 00:34:11 on dist-test-slave-gjw7"
server_key: "ef1389a0fc0cc51b05fff0f7d813f65a"
server_key_iv: "1c42f589d6f42eadf72e2ade5521cd17"
server_key_version: "encryptionkey@0"
I20260812 00:34:11.345796 13246 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 00:34:11.350229 13270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:11.351910 13246 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 00:34:11.352120 13246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "f1d7b2546608451ba09d6acafaa55c26"
format_stamp: "Formatted at 2026-08-12 00:34:11 on dist-test-slave-gjw7"
server_key: "ef1389a0fc0cc51b05fff0f7d813f65a"
server_key_iv: "1c42f589d6f42eadf72e2ade5521cd17"
server_key_version: "encryptionkey@0"
I20260812 00:34:11.352377 13246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:11.375541 13246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:11.380488 13246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:11.380808 13246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:11.382007 13246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:11.383586 13246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:11.383685 13246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:11.383817 13246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:11.383872 13246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:11.418291 13246 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:36457
I20260812 00:34:11.418365 13500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:36457 every 8 connection(s)
I20260812 00:34:11.419620 13246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:11.423530 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 13246
I20260812 00:34:11.423789 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:11.424691 8145 external_mini_cluster.cc:1428] Setting key c539a38ad626ef312fd5daddf239dc70
Aug 12 00:34:11 dist-test-slave-gjw7 krb5kdc[13042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:11.445001 13504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:11.422253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:46359 (local address 127.7.244.126:34705)
0812 00:34:11.422742 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:11.422749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:11.424023 (+ 1274us) server_negotiation.cc:407] Connection header received
0812 00:34:11.426482 (+ 2459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:11.426487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:11.426570 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:11.426688 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:11.429089 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:11.429911 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:11.432360 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:11.432811 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:11.432938 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:11.437658 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:11.437665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:11.437669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:11.437720 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:11.441128 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:11.442171 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:11.442178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:11.442181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:11.442378 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:11.442952 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:11.442957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:11.442960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:11.443343 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:11.443512 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:11.444339 (+ 827us) server_negotiation.cc:299] Negotiation successful
0812 00:34:11.444617 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":216,"threads_started":1}
I20260812 00:34:11.445958 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:11.422874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:34705 (local address 127.7.244.65:46359)
0812 00:34:11.423825 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:11.423881 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:11.426178 (+ 2297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:11.427015 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:11.427029 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:11.427867 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:11.428775 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:11.428833 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:11.430109 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:11.430114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:11.432100 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:11.432123 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:11.433099 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:11.433107 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:11.433465 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:11.434316 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:11.434349 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:11.437254 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:11.441417 (+ 4163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:11.441427 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:11.441444 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:11.441997 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:11.442549 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:11.442554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:11.442556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:11.442813 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:11.443492 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:11.443500 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:11.443900 (+ 400us) client_negotiation.cc:769] Sending connection context
0812 00:34:11.444154 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 00:34:11.444727 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":306,"threads_started":1}
I20260812 00:34:11.447607 13501 heartbeater.cc:347] Connected to a master server at 127.7.244.126:34705
I20260812 00:34:11.448046 13501 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:34:11.448899 13501 heartbeater.cc:511] Master 127.7.244.126:34705 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
I20260812 00:34:11.451891 13124 ts_manager.cc:194] Registered new tserver with Master: f1d7b2546608451ba09d6acafaa55c26 (127.7.244.65:36457)
I20260812 00:34:11.454018 13124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:46359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:34:11.482007 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:34705
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:11.837251 13508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:11.837647 13508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:11.837793 13508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:11.845420 13508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:11.845566 13508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:11.845726 13508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:11.856282 13508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:11.857918 13508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:11.859333 13508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:11.873046 13519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:11.873095 13520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:11.873116 13524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:11.874358 13508 server_base.cc:1034] running on GCE node
I20260812 00:34:11.875062 13508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:11.876102 13508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:11.877355 13508 hybrid_clock.cc:648] HybridClock initialized: now 1786494851877268 us; error 46 us; skew 500 ppm
I20260812 00:34:11.877718 13508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:11 dist-test-slave-gjw7 krb5kdc[13042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:11.881938 13508 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:34:11.884140 13508 webserver.cc:460] Webserver started at http://127.7.244.66:34017/ using document root <none> and password file <none>
I20260812 00:34:11.885052 13508 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:11.885262 13508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:11.885747 13508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:11.889123 13508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "12362c10b20349b89cee575f04b0e8c0"
format_stamp: "Formatted at 2026-08-12 00:34:11 on dist-test-slave-gjw7"
server_key: "6f57502c8f1e6085c884e376bc0c4381"
server_key_iv: "08c08fbfbcc7f16a358ee25880999a22"
server_key_version: "encryptionkey@0"
I20260812 00:34:11.890122 13508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "12362c10b20349b89cee575f04b0e8c0"
format_stamp: "Formatted at 2026-08-12 00:34:11 on dist-test-slave-gjw7"
server_key: "6f57502c8f1e6085c884e376bc0c4381"
server_key_iv: "08c08fbfbcc7f16a358ee25880999a22"
server_key_version: "encryptionkey@0"
I20260812 00:34:11.896055 13508 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260812 00:34:11.900089 13537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:11.901767 13508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 00:34:11.901950 13508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "12362c10b20349b89cee575f04b0e8c0"
format_stamp: "Formatted at 2026-08-12 00:34:11 on dist-test-slave-gjw7"
server_key: "6f57502c8f1e6085c884e376bc0c4381"
server_key_iv: "08c08fbfbcc7f16a358ee25880999a22"
server_key_version: "encryptionkey@0"
I20260812 00:34:11.902179 13508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:11.929607 13508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:11.934211 13508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:11.934525 13508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:11.935559 13508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:11.937162 13508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:11.937248 13508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:11.937397 13508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:11.937458 13508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:11.970448 13508 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:46427
I20260812 00:34:11.970525 13762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:46427 every 8 connection(s)
I20260812 00:34:11.971840 13508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:11.976052 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 13508
I20260812 00:34:11.976276 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:11.976926 8145 external_mini_cluster.cc:1428] Setting key 457d7a06a5344aafe2aec95c962669ab
Aug 12 00:34:11 dist-test-slave-gjw7 krb5kdc[13042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494851, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:11.994709 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:11.974581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:34521 (local address 127.7.244.126:34705)
0812 00:34:11.974996 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:11.975001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:11.976181 (+ 1180us) server_negotiation.cc:407] Connection header received
0812 00:34:11.978558 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:11.978564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:11.978647 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:11.978740 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:11.980944 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:11.981746 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:11.983915 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:11.984342 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:11.984455 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:11.988448 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:11.988455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:11.988460 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:11.988507 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:11.991256 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:11.992123 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:11.992128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:11.992132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:11.992274 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:11.992742 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:11.992747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:11.992749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:11.993064 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:11.993204 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:11.994049 (+ 845us) server_negotiation.cc:299] Negotiation successful
0812 00:34:11.994329 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":210,"threads_started":1}
I20260812 00:34:11.995709 13773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:11.975048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:34705 (local address 127.7.244.66:34521)
0812 00:34:11.975945 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:11.976012 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:11.978252 (+ 2240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:11.979046 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:11.979058 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:11.979864 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:11.980719 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:11.980748 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:11.981907 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:11.981912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:11.983762 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:11.983785 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:11.984576 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:11.984583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:11.984878 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:11.985667 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:11.985696 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:11.988178 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:11.991451 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:11.991464 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:11.991481 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:11.991985 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:11.992400 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:11.992405 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:11.992409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:11.992622 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:11.993201 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:11.993211 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:11.993646 (+ 435us) client_negotiation.cc:769] Sending connection context
0812 00:34:11.993946 (+ 300us) client_negotiation.cc:240] Negotiation successful
0812 00:34:11.994540 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":303,"threads_started":1}
I20260812 00:34:11.997522 13765 heartbeater.cc:347] Connected to a master server at 127.7.244.126:34705
I20260812 00:34:11.997865 13765 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
I20260812 00:34:11.998646 13765 heartbeater.cc:511] Master 127.7.244.126:34705 requested a full tablet report, sending...
I20260812 00:34:12.000463 13124 ts_manager.cc:194] Registered new tserver with Master: 12362c10b20349b89cee575f04b0e8c0 (127.7.244.66:46427)
I20260812 00:34:12.001380 13124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:34521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:34:12.029933 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:34705
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:12.380962 13778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:12.381260 13778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:12.381410 13778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:12.388865 13778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:12.389024 13778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:12.389151 13778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:12.399524 13778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:34705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:12.400996 13778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:12.402441 13778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:12.416982 13791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:12.417140 13789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:12.417179 13793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:12.417928 13778 server_base.cc:1034] running on GCE node
I20260812 00:34:12.418625 13778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:12.419696 13778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:12.420950 13778 hybrid_clock.cc:648] HybridClock initialized: now 1786494852420883 us; error 68 us; skew 500 ppm
I20260812 00:34:12.421396 13778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:12 dist-test-slave-gjw7 krb5kdc[13042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494852, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:12.425082 13778 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:34:12.426837 13778 webserver.cc:460] Webserver started at http://127.7.244.67:46391/ using document root <none> and password file <none>
I20260812 00:34:12.427688 13778 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:12.427826 13778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:12.428153 13778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:12.431877 13778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "9e1843e8fa7242ca9e12fc25a5594a45"
format_stamp: "Formatted at 2026-08-12 00:34:12 on dist-test-slave-gjw7"
server_key: "fade40467fe5f8e6bc8dc8271dc7828b"
server_key_iv: "65056556c742fe3f7556699f1c262221"
server_key_version: "encryptionkey@0"
I20260812 00:34:12.433224 13778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "9e1843e8fa7242ca9e12fc25a5594a45"
format_stamp: "Formatted at 2026-08-12 00:34:12 on dist-test-slave-gjw7"
server_key: "fade40467fe5f8e6bc8dc8271dc7828b"
server_key_iv: "65056556c742fe3f7556699f1c262221"
server_key_version: "encryptionkey@0"
I20260812 00:34:12.443536 13778 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.000s sys 0.008s
I20260812 00:34:12.447654 13801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:12.449141 13778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:34:12.449368 13778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "9e1843e8fa7242ca9e12fc25a5594a45"
format_stamp: "Formatted at 2026-08-12 00:34:12 on dist-test-slave-gjw7"
server_key: "fade40467fe5f8e6bc8dc8271dc7828b"
server_key_iv: "65056556c742fe3f7556699f1c262221"
server_key_version: "encryptionkey@0"
I20260812 00:34:12.449613 13778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:12.457638 13501 heartbeater.cc:503] Master 127.7.244.126:34705 was elected leader, sending a full tablet report...
I20260812 00:34:12.467710 13778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:12.472545 13778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:12.472856 13778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:12.474128 13778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:12.475700 13778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:12.475792 13778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:12.475950 13778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:12.476007 13778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:12.511140 13778 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:38955
I20260812 00:34:12.511179 14036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:38955 every 8 connection(s)
I20260812 00:34:12.512638 13778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:12.513226 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 13778
I20260812 00:34:12.513463 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:12.514061 8145 external_mini_cluster.cc:1428] Setting key d0f46a6c55cfd2cc96a7e20d37eda8a1
Aug 12 00:34:12 dist-test-slave-gjw7 krb5kdc[13042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494852, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:12.538256 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:12.515910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:41759 (local address 127.7.244.126:34705)
0812 00:34:12.516550 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:12.516555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:12.517898 (+ 1343us) server_negotiation.cc:407] Connection header received
0812 00:34:12.520155 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:12.520161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:12.520260 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:12.520375 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:12.523044 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:12.523921 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:12.526521 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:12.526993 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:12.527117 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:12.531626 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:12.531632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:12.531635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:12.531674 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:12.534682 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:12.535625 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:12.535629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:12.535632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:12.535781 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:12.536268 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:12.536272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:12.536274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:12.536583 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:12.536734 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:12.537591 (+ 857us) server_negotiation.cc:299] Negotiation successful
0812 00:34:12.537851 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":834,"threads_started":1}
I20260812 00:34:12.539309 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:12.516668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:34705 (local address 127.7.244.67:41759)
0812 00:34:12.517685 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:12.517757 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:12.519750 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:12.521073 (+ 1323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:12.521088 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:12.521950 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:12.522743 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:12.522772 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:12.524236 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:12.524242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:12.526272 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:12.526295 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:12.527300 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:12.527307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:12.527652 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:12.528460 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:12.528493 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:12.531354 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:12.534908 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:12.534918 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:12.534936 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:12.535464 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:12.535906 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:12.535912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:12.535914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:12.536141 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:12.536719 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:12.536729 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:12.537141 (+ 412us) client_negotiation.cc:769] Sending connection context
0812 00:34:12.537457 (+ 316us) client_negotiation.cc:240] Negotiation successful
0812 00:34:12.538075 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":346,"threads_started":1}
I20260812 00:34:12.540966 14037 heartbeater.cc:347] Connected to a master server at 127.7.244.126:34705
I20260812 00:34:12.541359 14037 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:12.542197 14037 heartbeater.cc:511] Master 127.7.244.126:34705 requested a full tablet report, sending...
I20260812 00:34:12.544162 13121 ts_manager.cc:194] Registered new tserver with Master: 9e1843e8fa7242ca9e12fc25a5594a45 (127.7.244.67:38955)
I20260812 00:34:12.545140 13121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:41759
I20260812 00:34:12.557499 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 00:34:12.598956 14043 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:12.596439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.7.244.126:34705)
0812 00:34:12.596623 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:12.596627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:12.597098 (+ 471us) server_negotiation.cc:407] Connection header received
0812 00:34:12.597238 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:12.597242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:12.597356 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:12.597461 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:12.598719 (+ 1258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51468: BlockingRecv error: recv got EOF from 127.0.0.1:51468 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 00:34:12.602411 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13246
W20260812 00:34:12.605472 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13246: Not authorized: Client connection negotiation failed: client connection to 127.7.244.65:36457: server requires authentication, but client does not have Kerberos credentials available
I20260812 00:34:12.605675 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 13246
I20260812 00:34:12.630972 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13508
W20260812 00:34:12.633867 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13508: Not authorized: Client connection negotiation failed: client connection to 127.7.244.66:46427: server requires authentication, but client does not have Kerberos credentials available
I20260812 00:34:12.634032 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 13508
I20260812 00:34:12.663501 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13778
W20260812 00:34:12.666752 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13778: Not authorized: Client connection negotiation failed: client connection to 127.7.244.67:38955: server requires authentication, but client does not have Kerberos credentials available
I20260812 00:34:12.666894 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 13778
I20260812 00:34:12.692366 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13064
I20260812 00:34:12.693819 13186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:12.798743 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 13064
2026-08-12T00:34:12Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4074 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:34:12 dist-test-slave-gjw7 krb5kdc[14082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:34:12 dist-test-slave-gjw7 krb5kdc[14082](info): set up 2 sockets
Aug 12 00:34:12 dist-test-slave-gjw7 krb5kdc[14082](info): commencing operation
krb5kdc: starting...
W20260812 00:34:14.116439 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:14 dist-test-slave-gjw7 krb5kdc[14082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:34:14.350224 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39683
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:39683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:14.705468 14105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:14.705796 14105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:14.705886 14105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:14.713316 14105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:14.713459 14105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:14.713510 14105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:14.713564 14105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:14.713616 14105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:14.724079 14105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:39683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:14.725530 14105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:14.727010 14105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:14.739835 14117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:14.739861 14115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:14.740473 14119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:14.740732 14105 server_base.cc:1034] running on GCE node
I20260812 00:34:14.741720 14105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:14.743392 14105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:14.744660 14105 hybrid_clock.cc:648] HybridClock initialized: now 1786494854744604 us; error 45 us; skew 500 ppm
I20260812 00:34:14.745108 14105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:14 dist-test-slave-gjw7 krb5kdc[14082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494854, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:14.754750 14105 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:34:14.756824 14105 webserver.cc:460] Webserver started at http://127.7.244.126:40395/ using document root <none> and password file <none>
I20260812 00:34:14.757702 14105 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:14.757834 14105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:14.758134 14105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:14.761497 14105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6"
format_stamp: "Formatted at 2026-08-12 00:34:14 on dist-test-slave-gjw7"
server_key: "284116c676894ba4a26eea17f76cfba1"
server_key_iv: "d1eea44ad9cec5fa4355dabd09748be9"
server_key_version: "encryptionkey@0"
I20260812 00:34:14.762372 14105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6"
format_stamp: "Formatted at 2026-08-12 00:34:14 on dist-test-slave-gjw7"
server_key: "284116c676894ba4a26eea17f76cfba1"
server_key_iv: "d1eea44ad9cec5fa4355dabd09748be9"
server_key_version: "encryptionkey@0"
I20260812 00:34:14.768864 14105 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:34:14.773152 14134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:14.774915 14105 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260812 00:34:14.775133 14105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6"
format_stamp: "Formatted at 2026-08-12 00:34:14 on dist-test-slave-gjw7"
server_key: "284116c676894ba4a26eea17f76cfba1"
server_key_iv: "d1eea44ad9cec5fa4355dabd09748be9"
server_key_version: "encryptionkey@0"
I20260812 00:34:14.775369 14105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:14.808130 14105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:14.813337 14105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:14.813612 14105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:14.832314 14105 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:39683
I20260812 00:34:14.832340 14231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:39683 every 8 connection(s)
I20260812 00:34:14.833972 14105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:14.835098 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 14105
I20260812 00:34:14.835330 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:14.836161 8145 external_mini_cluster.cc:1428] Setting key 026b3cec5ca3618e8844c03ddd46d18b
I20260812 00:34:14.839526 14234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 00:34:14 dist-test-slave-gjw7 krb5kdc[14082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:14.851603 14234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6: Bootstrap starting.
I20260812 00:34:14.855432 14234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:14.856649 14234 log.cc:824] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:14.859859 14234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6: No bootstrap required, opened a new log
I20260812 00:34:14.860642 14241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:14.838552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.7.244.126:39683)
0812 00:34:14.839463 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:14.839477 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:14.839527 (+ 50us) server_negotiation.cc:407] Connection header received
0812 00:34:14.841330 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:14.841367 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:14.841813 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:14.842265 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:14.843924 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:14.845585 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:14.846981 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:14.847726 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:14.847958 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:14.852446 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:14.852456 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:14.852475 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:14.852524 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:14.855903 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:14.856816 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:14.856828 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:14.856838 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:14.857061 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:14.857540 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:14.857544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:14.857546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:14.858168 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:14.858503 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:14.859012 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 00:34:14.859481 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":407,"threads_started":1}
I20260812 00:34:14.865085 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:34:14.868464 14234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39683 } }
I20260812 00:34:14.868873 14234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:14.868970 14234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7ce38f096e42d1a3630cf4f6e0c2e6, State: Initialized, Role: FOLLOWER
I20260812 00:34:14.869637 14234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39683 } }
I20260812 00:34:14.869903 14234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:14.870039 14234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:14.870208 14234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:14.872491 14234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39683 } }
I20260812 00:34:14.873179 14234 leader_election.cc:302] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c7ce38f096e42d1a3630cf4f6e0c2e6; no voters:
I20260812 00:34:14.873984 14234 leader_election.cc:288] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:14.874110 14246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:14.875392 14246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [term 1 LEADER]: Becoming Leader. State: Replica: 4c7ce38f096e42d1a3630cf4f6e0c2e6, State: Running, Role: LEADER
I20260812 00:34:14.875994 14246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39683 } }
I20260812 00:34:14.876477 14234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:14.878919 14249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39683 } } }
I20260812 00:34:14.879350 14249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:14.880678 14255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:14.880168 14251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c7ce38f096e42d1a3630cf4f6e0c2e6. Latest consensus state: current_term: 1 leader_uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c7ce38f096e42d1a3630cf4f6e0c2e6" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39683 } } }
I20260812 00:34:14.882081 14251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:14.887413 14255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:14.909789 14255 catalog_manager.cc:1347] Generated new cluster ID: b00da484a826417f905f92c7a57dffff
I20260812 00:34:14.909940 14255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:14.922323 14279 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 00:34:14.931658 14255 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:14.933126 14255 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:14.936507 14241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:14.925888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.7.244.126:39683)
0812 00:34:14.926106 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:14.926114 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:14.926703 (+ 589us) server_negotiation.cc:407] Connection header received
0812 00:34:14.926908 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:14.926914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:14.926994 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:14.927087 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:14.928524 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:14.929489 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:14.931468 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:14.932146 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:14.932283 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:14.934688 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:14.934696 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:14.934701 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:14.935561 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:14.935646 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:14.935676 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:14.935788 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:14.935996 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 00:34:14.936145 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 00:34:14.944749 14255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c7ce38f096e42d1a3630cf4f6e0c2e6: Generated new TSK 0
I20260812 00:34:14.946213 14255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:34:14.956284 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14105
I20260812 00:34:14.957587 14220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:15.050287 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 14105
2026-08-12T00:34:15Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2246 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:34:15 dist-test-slave-gjw7 krb5kdc[14291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:34:15 dist-test-slave-gjw7 krb5kdc[14291](info): set up 2 sockets
Aug 12 00:34:15 dist-test-slave-gjw7 krb5kdc[14291](info): commencing operation
krb5kdc: starting...
W20260812 00:34:16.366321 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:16 dist-test-slave-gjw7 krb5kdc[14291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:34:16.601408 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:46381
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:46381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:16.949702 14312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:16.950088 14312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:16.950228 14312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:16.957964 14312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:16.958114 14312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:16.958197 14312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:16.958277 14312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:16.958345 14312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:16.968964 14312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:46381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:46381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:16.970487 14312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:16.971915 14312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:16.984445 14319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:16.984424 14320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:16.984475 14325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:16.985445 14312 server_base.cc:1034] running on GCE node
I20260812 00:34:16.986305 14312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:16.987828 14312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:16.989084 14312 hybrid_clock.cc:648] HybridClock initialized: now 1786494856989030 us; error 37 us; skew 500 ppm
I20260812 00:34:16.989523 14312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:16 dist-test-slave-gjw7 krb5kdc[14291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494856, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:16.992923 14312 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:34:16.994764 14312 webserver.cc:460] Webserver started at http://127.7.244.126:43007/ using document root <none> and password file <none>
I20260812 00:34:16.995532 14312 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:16.995666 14312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:16.996016 14312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:16.999509 14312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3"
format_stamp: "Formatted at 2026-08-12 00:34:16 on dist-test-slave-gjw7"
server_key: "d5509c3b5a75693e55ca2ddc6ab549d9"
server_key_iv: "336545bfeee4ab281108d04f7182454a"
server_key_version: "encryptionkey@0"
I20260812 00:34:17.000404 14312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3"
format_stamp: "Formatted at 2026-08-12 00:34:16 on dist-test-slave-gjw7"
server_key: "d5509c3b5a75693e55ca2ddc6ab549d9"
server_key_iv: "336545bfeee4ab281108d04f7182454a"
server_key_version: "encryptionkey@0"
I20260812 00:34:17.006527 14312 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 00:34:17.011165 14341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:17.012856 14312 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260812 00:34:17.013099 14312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3"
format_stamp: "Formatted at 2026-08-12 00:34:16 on dist-test-slave-gjw7"
server_key: "d5509c3b5a75693e55ca2ddc6ab549d9"
server_key_iv: "336545bfeee4ab281108d04f7182454a"
server_key_version: "encryptionkey@0"
I20260812 00:34:17.013365 14312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:17.031440 14312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:17.036427 14312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:17.036752 14312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:17.055951 14312 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:46381
I20260812 00:34:17.055974 14445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:46381 every 8 connection(s)
I20260812 00:34:17.057487 14312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:17.061508 14446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:17.064934 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 14312
I20260812 00:34:17.065186 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:17.066062 8145 external_mini_cluster.cc:1428] Setting key ff7ab611705f43147fe007f6409f63f3
I20260812 00:34:17.074990 14446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3: Bootstrap starting.
I20260812 00:34:17.078392 14446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:17.079490 14446 log.cc:824] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:34:17 dist-test-slave-gjw7 krb5kdc[14291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:17.082878 14446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3: No bootstrap required, opened a new log
I20260812 00:34:17.090042 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:17.068556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.7.244.126:46381)
0812 00:34:17.069464 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:17.069478 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:17.069526 (+ 48us) server_negotiation.cc:407] Connection header received
0812 00:34:17.071285 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:17.071312 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:17.071764 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:17.072196 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:17.073768 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:17.075136 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:17.076426 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:17.077135 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:17.077342 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:17.081776 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:17.081786 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:17.081803 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:17.081858 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:17.085163 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:17.086070 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:17.086079 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:17.086087 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:17.086310 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:17.086737 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:17.086741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:17.086743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:17.087396 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:17.087748 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:17.088238 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 00:34:17.088737 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":373,"threads_started":1}
I20260812 00:34:17.091478 14446 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46381 } }
I20260812 00:34:17.091890 14446 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:17.092011 14446 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf7ceffedf2d4f7c91bbf12b7ede1dd3, State: Initialized, Role: FOLLOWER
I20260812 00:34:17.092658 14446 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46381 } }
I20260812 00:34:17.092912 14446 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:17.093080 14446 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:17.093284 14446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:17.095726 14446 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46381 } }
I20260812 00:34:17.096346 14446 leader_election.cc:302] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf7ceffedf2d4f7c91bbf12b7ede1dd3; no voters:
I20260812 00:34:17.097086 14446 leader_election.cc:288] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:17.097213 14456 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:17.098013 14456 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [term 1 LEADER]: Becoming Leader. State: Replica: cf7ceffedf2d4f7c91bbf12b7ede1dd3, State: Running, Role: LEADER
I20260812 00:34:17.098589 14456 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46381 } }
I20260812 00:34:17.099442 14446 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:17.101583 14459 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf7ceffedf2d4f7c91bbf12b7ede1dd3. Latest consensus state: current_term: 1 leader_uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46381 } } }
I20260812 00:34:17.102022 14459 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:17.105558 14457 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf7ceffedf2d4f7c91bbf12b7ede1dd3" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46381 } } }
I20260812 00:34:17.105886 14457 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:17.106508 14468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:17.111838 14468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:17.121747 14468 catalog_manager.cc:1347] Generated new cluster ID: 29912d19961c4a8bb9e3f5d22bce3363
I20260812 00:34:17.121889 14468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:17.145349 14468 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:17.147466 14468 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:17.155081 14468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf7ceffedf2d4f7c91bbf12b7ede1dd3: Generated new TSK 0
I20260812 00:34:17.156312 14468 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:34:17.217762 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46381
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:17.577744 14497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:17.578068 14497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:17.578191 14497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:17.586249 14497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:17.586393 14497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:17.586550 14497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:17.597543 14497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:17.599020 14497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:17.600494 14497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:17.615391 14507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:17.615435 14515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:17.615402 14518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:17.616699 14497 server_base.cc:1034] running on GCE node
I20260812 00:34:17.617527 14497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:17.618608 14497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:17.619843 14497 hybrid_clock.cc:648] HybridClock initialized: now 1786494857619795 us; error 52 us; skew 500 ppm
I20260812 00:34:17.620199 14497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:17 dist-test-slave-gjw7 krb5kdc[14291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:17.623773 14497 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:34:17.625595 14497 webserver.cc:460] Webserver started at http://127.7.244.65:35023/ using document root <none> and password file <none>
I20260812 00:34:17.626392 14497 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:17.626536 14497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:17.626864 14497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:17.630306 14497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "bf369efcf88141e9a31cca22ed894dc2"
format_stamp: "Formatted at 2026-08-12 00:34:17 on dist-test-slave-gjw7"
server_key: "0a7e977ffb0e0664a7049ae5626daae2"
server_key_iv: "57d79ecba0a79f7f416ea996e0a37422"
server_key_version: "encryptionkey@0"
I20260812 00:34:17.631218 14497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "bf369efcf88141e9a31cca22ed894dc2"
format_stamp: "Formatted at 2026-08-12 00:34:17 on dist-test-slave-gjw7"
server_key: "0a7e977ffb0e0664a7049ae5626daae2"
server_key_iv: "57d79ecba0a79f7f416ea996e0a37422"
server_key_version: "encryptionkey@0"
I20260812 00:34:17.637182 14497 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:34:17.641271 14527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:17.642882 14497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 00:34:17.643079 14497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "bf369efcf88141e9a31cca22ed894dc2"
format_stamp: "Formatted at 2026-08-12 00:34:17 on dist-test-slave-gjw7"
server_key: "0a7e977ffb0e0664a7049ae5626daae2"
server_key_iv: "57d79ecba0a79f7f416ea996e0a37422"
server_key_version: "encryptionkey@0"
I20260812 00:34:17.643288 14497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:17.660918 14497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:17.665604 14497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:17.665910 14497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:17.666919 14497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:17.668386 14497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:17.668470 14497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:17.668601 14497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:17.668654 14497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:17.700176 14497 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:38437
I20260812 00:34:17.700227 14773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:38437 every 8 connection(s)
I20260812 00:34:17.701644 14497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:17.711861 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 14497
I20260812 00:34:17.712076 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:17.712761 8145 external_mini_cluster.cc:1428] Setting key 2054bd55d1242c4e8d2eb0cf484780c8
Aug 12 00:34:17 dist-test-slave-gjw7 krb5kdc[14291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494857, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:17.725282 14778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:17.704387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:41843 (local address 127.7.244.126:46381)
0812 00:34:17.704929 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:17.704934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:17.706007 (+ 1073us) server_negotiation.cc:407] Connection header received
0812 00:34:17.708165 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:17.708172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:17.708277 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:17.708391 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:17.710449 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:17.711258 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:17.713157 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:17.713809 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:17.713964 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:17.718450 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:17.718459 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:17.718465 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:17.718514 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:17.721762 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:17.722770 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:17.722774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:17.722777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:17.722928 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:17.723461 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:17.723465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:17.723467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:17.723744 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:17.723857 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:17.724813 (+ 956us) server_negotiation.cc:299] Negotiation successful
0812 00:34:17.725020 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":284,"threads_started":1}
I20260812 00:34:17.726869 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:17.704954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46381 (local address 127.7.244.65:41843)
0812 00:34:17.705814 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:17.705868 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:17.707786 (+ 1918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:17.708716 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:17.708726 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:17.709476 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:17.710218 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:17.710242 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:17.711474 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:17.711479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:17.712961 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:17.712979 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:17.714079 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:17.714088 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:17.714396 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:17.715203 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:17.715232 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:17.718116 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:17.721972 (+ 3856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:17.721982 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:17.722000 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:17.722609 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:17.723038 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:17.723043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:17.723046 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:17.723343 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:17.723880 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:17.723888 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:17.724379 (+ 491us) client_negotiation.cc:769] Sending connection context
0812 00:34:17.724716 (+ 337us) client_negotiation.cc:240] Negotiation successful
0812 00:34:17.725360 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":339,"threads_started":1}
I20260812 00:34:17.728569 14774 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46381
I20260812 00:34:17.728896 14774 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:17.729759 14774 heartbeater.cc:511] Master 127.7.244.126:46381 requested a full tablet report, sending...
I20260812 00:34:17.732004 14372 ts_manager.cc:194] Registered new tserver with Master: bf369efcf88141e9a31cca22ed894dc2 (127.7.244.65:38437)
I20260812 00:34:17.733603 14372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:41843
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:34:17.765704 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46381
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:18.121675 14793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:18.122004 14793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:18.122107 14793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:18.129978 14793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:18.130115 14793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:18.130241 14793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:18.140568 14793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:18.142010 14793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:18.143414 14793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:18.157790 14813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:18.157958 14809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:18.157811 14816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:18.158736 14793 server_base.cc:1034] running on GCE node
I20260812 00:34:18.159413 14793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:18.160529 14793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:18.161752 14793 hybrid_clock.cc:648] HybridClock initialized: now 1786494858161694 us; error 50 us; skew 500 ppm
I20260812 00:34:18.162113 14793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:18.165570 14793 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:34:18.167380 14793 webserver.cc:460] Webserver started at http://127.7.244.66:43241/ using document root <none> and password file <none>
I20260812 00:34:18.168169 14793 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:18.168318 14793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:18.168654 14793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:18.171950 14793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "4be63bba11ec4c7bbf5f9ebc4c65ee0f"
format_stamp: "Formatted at 2026-08-12 00:34:18 on dist-test-slave-gjw7"
server_key: "da27c2830bc6a5b8dd2633fdee42927b"
server_key_iv: "417a02276690f3969c4016e93f856c4a"
server_key_version: "encryptionkey@0"
I20260812 00:34:18.172855 14793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "4be63bba11ec4c7bbf5f9ebc4c65ee0f"
format_stamp: "Formatted at 2026-08-12 00:34:18 on dist-test-slave-gjw7"
server_key: "da27c2830bc6a5b8dd2633fdee42927b"
server_key_iv: "417a02276690f3969c4016e93f856c4a"
server_key_version: "encryptionkey@0"
I20260812 00:34:18.178937 14793 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260812 00:34:18.183056 14827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:18.184717 14793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 00:34:18.184948 14793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "4be63bba11ec4c7bbf5f9ebc4c65ee0f"
format_stamp: "Formatted at 2026-08-12 00:34:18 on dist-test-slave-gjw7"
server_key: "da27c2830bc6a5b8dd2633fdee42927b"
server_key_iv: "417a02276690f3969c4016e93f856c4a"
server_key_version: "encryptionkey@0"
I20260812 00:34:18.185181 14793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:18.205698 14793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:18.210966 14793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:18.211283 14793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:18.212389 14793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:18.213913 14793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:18.213999 14793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:18.214133 14793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:18.214186 14793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:18.247306 14793 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:33635
I20260812 00:34:18.247341 15075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:33635 every 8 connection(s)
I20260812 00:34:18.248848 14793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:18.259187 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 14793
I20260812 00:34:18.259353 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:18.259821 8145 external_mini_cluster.cc:1428] Setting key f00de8a921ec8f92f70c19d7c468b851
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:18.270951 15079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:18.251655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:49161 (local address 127.7.244.126:46381)
0812 00:34:18.252132 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:18.252137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:18.253082 (+ 945us) server_negotiation.cc:407] Connection header received
0812 00:34:18.255023 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:18.255029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:18.255121 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:18.255226 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:18.256944 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.257770 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.259450 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.259936 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.260072 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:18.264345 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:18.264351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:18.264355 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:18.264395 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:18.267199 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.268281 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.268285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.268288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.268444 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.268958 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.268961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.268963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.269250 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:18.269448 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:18.270272 (+ 824us) server_negotiation.cc:299] Negotiation successful
0812 00:34:18.270539 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":235,"threads_started":1}
I20260812 00:34:18.272069 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:18.252109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46381 (local address 127.7.244.66:49161)
0812 00:34:18.252882 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:18.252934 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:18.254697 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:18.255520 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:18.255530 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:18.256106 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:18.256727 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:18.256747 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.257944 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.257948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:18.259282 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:18.259301 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.260200 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.260208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:18.260472 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:18.261180 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:18.261211 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:18.264018 (+ 2807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:18.267436 (+ 3418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:18.267448 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:18.267471 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:18.268148 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:18.268560 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:18.268564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:18.268567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:18.268850 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:18.269448 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:18.269458 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:18.269862 (+ 404us) client_negotiation.cc:769] Sending connection context
0812 00:34:18.270153 (+ 291us) client_negotiation.cc:240] Negotiation successful
0812 00:34:18.270787 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":296,"threads_started":1}
I20260812 00:34:18.273876 15076 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46381
I20260812 00:34:18.274214 15076 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:18.275035 15076 heartbeater.cc:511] Master 127.7.244.126:46381 requested a full tablet report, sending...
I20260812 00:34:18.276883 14372 ts_manager.cc:194] Registered new tserver with Master: 4be63bba11ec4c7bbf5f9ebc4c65ee0f (127.7.244.66:33635)
I20260812 00:34:18.278101 14372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:49161
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:34:18.313582 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46381
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:18.665788 15083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:18.666109 15083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:18.666250 15083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:18.673667 15083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:18.673797 15083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:18.673954 15083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:18.684407 15083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:18.685909 15083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:18.687240 15083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:18.701404 15099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:18.701421 15095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:18.702325 15083 server_base.cc:1034] running on GCE node
W20260812 00:34:18.706223 15094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:18.707056 15083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:18.708104 15083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:18.709327 15083 hybrid_clock.cc:648] HybridClock initialized: now 1786494858709251 us; error 52 us; skew 500 ppm
I20260812 00:34:18.709668 15083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:18.713081 15083 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:34:18.714811 15083 webserver.cc:460] Webserver started at http://127.7.244.67:36493/ using document root <none> and password file <none>
I20260812 00:34:18.715644 15083 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:18.715770 15083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:18.716133 15083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:18.719619 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "d7bab6fa181a4d82972f6eeb0b4d5cbd"
format_stamp: "Formatted at 2026-08-12 00:34:18 on dist-test-slave-gjw7"
server_key: "ecc5ecd7d098d7e7d706f0454a40f643"
server_key_iv: "a3df80d56f8483830c1e93aa74afee4c"
server_key_version: "encryptionkey@0"
I20260812 00:34:18.720661 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "d7bab6fa181a4d82972f6eeb0b4d5cbd"
format_stamp: "Formatted at 2026-08-12 00:34:18 on dist-test-slave-gjw7"
server_key: "ecc5ecd7d098d7e7d706f0454a40f643"
server_key_iv: "a3df80d56f8483830c1e93aa74afee4c"
server_key_version: "encryptionkey@0"
I20260812 00:34:18.726717 15083 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 00:34:18.730798 15114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:18.732362 15083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:34:18.732545 15083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "d7bab6fa181a4d82972f6eeb0b4d5cbd"
format_stamp: "Formatted at 2026-08-12 00:34:18 on dist-test-slave-gjw7"
server_key: "ecc5ecd7d098d7e7d706f0454a40f643"
server_key_iv: "a3df80d56f8483830c1e93aa74afee4c"
server_key_version: "encryptionkey@0"
I20260812 00:34:18.732765 15083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:18.736881 14774 heartbeater.cc:503] Master 127.7.244.126:46381 was elected leader, sending a full tablet report...
I20260812 00:34:18.753372 15083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:18.758291 15083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:18.758639 15083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:18.759720 15083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:18.761245 15083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:18.761358 15083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 00:34:18.761504 15083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:18.761562 15083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:18.794219 15083 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:40761
I20260812 00:34:18.794260 15362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:40761 every 8 connection(s)
I20260812 00:34:18.795686 15083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:18.797559 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 15083
I20260812 00:34:18.797758 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:18.798337 8145 external_mini_cluster.cc:1428] Setting key c6efc6fdfab2fdcdfd2cda6f606adc69
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:18.818683 15370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:18.798892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:57565 (local address 127.7.244.126:46381)
0812 00:34:18.799425 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:18.799431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:18.800834 (+ 1403us) server_negotiation.cc:407] Connection header received
0812 00:34:18.803167 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:18.803172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:18.803253 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:18.803349 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:18.805133 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.806031 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.807821 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.808268 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.808385 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:18.812356 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:18.812363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:18.812366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:18.812407 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:18.815607 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.816438 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.816441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.816443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.816565 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.816970 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.816974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.816977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.817275 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:18.817464 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:18.818184 (+ 720us) server_negotiation.cc:299] Negotiation successful
0812 00:34:18.818376 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":548,"threads_started":1}
I20260812 00:34:18.819730 15372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:18.799577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46381 (local address 127.7.244.67:57565)
0812 00:34:18.800655 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:18.800714 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:18.802848 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:18.803615 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:18.803625 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:18.804206 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:18.804897 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:18.804920 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.806229 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.806235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:18.807605 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:18.807622 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.808558 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.808565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:18.808854 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:18.809680 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:18.809711 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:18.812097 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:18.815759 (+ 3662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:18.815769 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:18.815786 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:18.816311 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:18.816672 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:18.816676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:18.816678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:18.816883 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:18.817427 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:18.817433 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:18.817818 (+ 385us) client_negotiation.cc:769] Sending connection context
0812 00:34:18.818075 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 00:34:18.818534 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":391,"threads_started":1}
I20260812 00:34:18.821285 15363 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46381
I20260812 00:34:18.821636 15363 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:18.822420 15363 heartbeater.cc:511] Master 127.7.244.126:46381 requested a full tablet report, sending...
I20260812 00:34:18.824162 14372 ts_manager.cc:194] Registered new tserver with Master: d7bab6fa181a4d82972f6eeb0b4d5cbd (127.7.244.67:40761)
I20260812 00:34:18.825085 14372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:57565
I20260812 00:34:18.827620 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:18.884770 15370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:18.869656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59458 (local address 127.7.244.126:46381)
0812 00:34:18.869840 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:18.869846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:18.870331 (+ 485us) server_negotiation.cc:407] Connection header received
0812 00:34:18.870535 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:18.870539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:18.870616 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:18.870706 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:18.872244 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.873029 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.874427 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.874876 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.875004 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:18.879321 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:18.879327 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:18.879331 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:18.879376 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:18.881993 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.882743 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.882747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.882750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.882885 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.883287 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.883291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.883293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.883648 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:18.883893 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:18.884222 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 00:34:18.884474 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 00:34:18.885347 14372 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59458
I20260812 00:34:18.887327 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14497
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:34:18.905485 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:18.888516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38432 (local address 127.7.244.65:38437)
0812 00:34:18.889124 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:18.889131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:18.889159 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:34:18.889224 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:18.889230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:18.889550 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:18.889693 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:18.891086 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.892184 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.893559 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.894290 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.894427 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:18.898976 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:18.898994 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:18.899012 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:18.899066 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:18.902173 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.903011 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.903020 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.903022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.903194 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.903665 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.903671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.903673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.904119 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:18.904371 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:18.904750 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 00:34:18.905036 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":276,"threads_started":1}
W20260812 00:34:18.906836 14769 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38432
W20260812 00:34:18.908052 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14497: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:34:18.908265 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 14497
I20260812 00:34:18.935877 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14793
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:34:18.953433 15396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:18.936853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40428 (local address 127.7.244.66:33635)
0812 00:34:18.937526 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:18.937535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:18.937562 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:34:18.937651 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:18.937661 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:18.937911 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:18.938094 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:18.939498 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.940601 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.942039 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.942740 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.942852 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:18.947224 (+ 4372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:18.947241 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:18.947259 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:18.947309 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:18.950416 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.951152 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.951159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.951161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.951296 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.951703 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.951706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.951708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.952079 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:18.952316 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:18.952745 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 00:34:18.953053 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":340,"threads_started":1}
W20260812 00:34:18.954764 15069 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40428
W20260812 00:34:18.955968 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14793: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:34:18.956130 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 14793
I20260812 00:34:18.980768 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 15083
Aug 12 00:34:18 dist-test-slave-gjw7 krb5kdc[14291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494858, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:34:18.998878 15398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:18.981981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37616 (local address 127.7.244.67:40761)
0812 00:34:18.982795 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:18.982806 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:18.982846 (+ 40us) server_negotiation.cc:407] Connection header received
0812 00:34:18.982928 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:18.982939 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:18.983167 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:18.983337 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:18.984807 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.986049 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.987402 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:18.988027 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:18.988158 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:18.992857 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:18.992873 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:18.992890 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:18.992937 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:18.995710 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.996516 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.996525 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.996528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.996672 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:18.997087 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:18.997091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:18.997093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:18.997548 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:18.997871 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:18.998216 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 00:34:18.998522 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":410,"threads_started":1}
W20260812 00:34:19.000279 15350 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37616
W20260812 00:34:19.001538 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 15083: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:34:19.001765 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 15083
I20260812 00:34:19.042387 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14312
I20260812 00:34:19.043787 14437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:19.148597 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 14312
2026-08-12T00:34:19Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4105 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:34:19 dist-test-slave-gjw7 krb5kdc[15410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:34:19 dist-test-slave-gjw7 krb5kdc[15410](info): set up 2 sockets
Aug 12 00:34:19 dist-test-slave-gjw7 krb5kdc[15410](info): commencing operation
krb5kdc: starting...
W20260812 00:34:20.461733 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:20 dist-test-slave-gjw7 krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:34:20.704475 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39451
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:39451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:21.059002 15434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:21.059360 15434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:21.059504 15434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:21.067174 15434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:21.067323 15434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:21.067405 15434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:21.067484 15434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:21.067548 15434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:21.078052 15434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:39451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:21.079665 15434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:21.081127 15434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:21.093843 15450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:21.093961 15446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:21.093924 15447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:21.094533 15434 server_base.cc:1034] running on GCE node
I20260812 00:34:21.095494 15434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:21.097206 15434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:21.098482 15434 hybrid_clock.cc:648] HybridClock initialized: now 1786494861098423 us; error 43 us; skew 500 ppm
I20260812 00:34:21.098908 15434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:21 dist-test-slave-gjw7 krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:21.106060 15434 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:34:21.107832 15434 webserver.cc:460] Webserver started at http://127.7.244.126:45451/ using document root <none> and password file <none>
I20260812 00:34:21.108611 15434 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:21.108748 15434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:21.109103 15434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:21.112376 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "50149d867934450db5c127d6c8ecd6b9"
format_stamp: "Formatted at 2026-08-12 00:34:21 on dist-test-slave-gjw7"
server_key: "ec2ad82b59f794bde738506ece31772a"
server_key_iv: "0510a50f78f1e00faf1c4cc720fa531d"
server_key_version: "encryptionkey@0"
I20260812 00:34:21.113336 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "50149d867934450db5c127d6c8ecd6b9"
format_stamp: "Formatted at 2026-08-12 00:34:21 on dist-test-slave-gjw7"
server_key: "ec2ad82b59f794bde738506ece31772a"
server_key_iv: "0510a50f78f1e00faf1c4cc720fa531d"
server_key_version: "encryptionkey@0"
I20260812 00:34:21.118970 15434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 00:34:21.122794 15464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:21.124382 15434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:34:21.124574 15434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "50149d867934450db5c127d6c8ecd6b9"
format_stamp: "Formatted at 2026-08-12 00:34:21 on dist-test-slave-gjw7"
server_key: "ec2ad82b59f794bde738506ece31772a"
server_key_iv: "0510a50f78f1e00faf1c4cc720fa531d"
server_key_version: "encryptionkey@0"
I20260812 00:34:21.124788 15434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:21.139812 15434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:21.144099 15434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:21.144399 15434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:21.163327 15434 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:39451
I20260812 00:34:21.163363 15573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:39451 every 8 connection(s)
I20260812 00:34:21.164963 15434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:21.168475 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 15434
I20260812 00:34:21.168730 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:21.169615 8145 external_mini_cluster.cc:1428] Setting key c600f20173ddbe97cd127a44e41b5d00
I20260812 00:34:21.170002 15574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:21.183717 15574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9: Bootstrap starting.
Aug 12 00:34:21 dist-test-slave-gjw7 krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:21.187341 15574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:21.188683 15574 log.cc:824] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:21.191875 15574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9: No bootstrap required, opened a new log
I20260812 00:34:21.194593 15583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:21.171947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33912 (local address 127.7.244.126:39451)
0812 00:34:21.172754 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:21.172767 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:21.172810 (+ 43us) server_negotiation.cc:407] Connection header received
0812 00:34:21.175253 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:21.175282 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:21.175791 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:21.176277 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:21.178017 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:21.179384 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:21.180775 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:21.181664 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:21.181864 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:21.186662 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:21.186681 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:21.186699 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:21.186752 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:21.190128 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:21.190956 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:21.190963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:21.190971 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:21.191154 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:21.191562 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:21.191566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:21.191568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:21.192184 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:21.192484 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:21.192959 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 00:34:21.193447 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":358,"threads_started":1}
I20260812 00:34:21.200433 15574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50149d867934450db5c127d6c8ecd6b9" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39451 } }
I20260812 00:34:21.200872 15574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:21.200981 15574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50149d867934450db5c127d6c8ecd6b9, State: Initialized, Role: FOLLOWER
I20260812 00:34:21.201721 15574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50149d867934450db5c127d6c8ecd6b9" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39451 } }
I20260812 00:34:21.202046 15574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:21.202203 15574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:21.202378 15574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:21.204777 15574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50149d867934450db5c127d6c8ecd6b9" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39451 } }
I20260812 00:34:21.205443 15574 leader_election.cc:302] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50149d867934450db5c127d6c8ecd6b9; no voters:
I20260812 00:34:21.206096 15574 leader_election.cc:288] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:21.206215 15585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:21.207333 15585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [term 1 LEADER]: Becoming Leader. State: Replica: 50149d867934450db5c127d6c8ecd6b9, State: Running, Role: LEADER
I20260812 00:34:21.207952 15585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50149d867934450db5c127d6c8ecd6b9" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39451 } }
I20260812 00:34:21.208490 15574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:21.210886 15588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50149d867934450db5c127d6c8ecd6b9. Latest consensus state: current_term: 1 leader_uuid: "50149d867934450db5c127d6c8ecd6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50149d867934450db5c127d6c8ecd6b9" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39451 } } }
I20260812 00:34:21.211325 15588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:21.211282 15587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50149d867934450db5c127d6c8ecd6b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50149d867934450db5c127d6c8ecd6b9" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39451 } } }
I20260812 00:34:21.211555 15587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:21.212477 15600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:21.218740 15600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:21.228641 15600 catalog_manager.cc:1347] Generated new cluster ID: 1d138aa36d8446dcb1d873f4383d4ed9
I20260812 00:34:21.228788 15600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:21.252321 15600 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:21.253813 15600 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:21.264958 15600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50149d867934450db5c127d6c8ecd6b9: Generated new TSK 0
I20260812 00:34:21.266036 15600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:34:21.329643 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39451
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:21.687336 15636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:21.687674 15636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:21.687774 15636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:21.696043 15636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:21.696173 15636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:21.696290 15636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:21.706578 15636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:21.707980 15636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:21.709512 15636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:21.723869 15653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:21.723948 15651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:21.723933 15650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:21.724648 15636 server_base.cc:1034] running on GCE node
I20260812 00:34:21.725365 15636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:21.726408 15636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:21.727627 15636 hybrid_clock.cc:648] HybridClock initialized: now 1786494861727567 us; error 53 us; skew 500 ppm
I20260812 00:34:21.727954 15636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:21 dist-test-slave-gjw7 krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:21.731511 15636 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:34:21.733435 15636 webserver.cc:460] Webserver started at http://127.7.244.65:41619/ using document root <none> and password file <none>
I20260812 00:34:21.734220 15636 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:21.734361 15636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:21.734738 15636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:21.738227 15636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "746dbe1f5c434cd29182b253fe9bd1a1"
format_stamp: "Formatted at 2026-08-12 00:34:21 on dist-test-slave-gjw7"
server_key: "8113202b3e427e2f09d644a1b12742c4"
server_key_iv: "24bee6801ade669b3ce77bf88d0eca53"
server_key_version: "encryptionkey@0"
I20260812 00:34:21.739157 15636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "746dbe1f5c434cd29182b253fe9bd1a1"
format_stamp: "Formatted at 2026-08-12 00:34:21 on dist-test-slave-gjw7"
server_key: "8113202b3e427e2f09d644a1b12742c4"
server_key_iv: "24bee6801ade669b3ce77bf88d0eca53"
server_key_version: "encryptionkey@0"
I20260812 00:34:21.745354 15636 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:34:21.749394 15671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:21.750934 15636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 00:34:21.751117 15636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "746dbe1f5c434cd29182b253fe9bd1a1"
format_stamp: "Formatted at 2026-08-12 00:34:21 on dist-test-slave-gjw7"
server_key: "8113202b3e427e2f09d644a1b12742c4"
server_key_iv: "24bee6801ade669b3ce77bf88d0eca53"
server_key_version: "encryptionkey@0"
I20260812 00:34:21.751328 15636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:21.769580 15636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:21.773937 15636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:21.774241 15636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:21.775318 15636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:21.776820 15636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:21.776908 15636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:21.777045 15636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:21.777094 15636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:21.809523 15636 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:41047
I20260812 00:34:21.809538 15911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:41047 every 8 connection(s)
I20260812 00:34:21.811009 15636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:21.814055 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 15636
I20260812 00:34:21.814325 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:21.815182 8145 external_mini_cluster.cc:1428] Setting key ab390a011468540523fc6e8b9b0d68ee
Aug 12 00:34:21 dist-test-slave-gjw7 krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494861, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:21.833269 15921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:21.813928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:46865 (local address 127.7.244.126:39451)
0812 00:34:21.814467 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:21.814473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:21.815749 (+ 1276us) server_negotiation.cc:407] Connection header received
0812 00:34:21.817900 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:21.817906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:21.818009 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:21.818105 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:21.819816 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:21.820575 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:21.822306 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:21.822732 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:21.822863 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:21.827114 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:21.827120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:21.827123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:21.827162 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:21.829936 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:21.830787 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:21.830791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:21.830793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:21.830934 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:21.831391 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:21.831395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:21.831397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:21.831685 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:21.831840 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:21.832639 (+ 799us) server_negotiation.cc:299] Negotiation successful
0812 00:34:21.832886 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":280,"threads_started":1}
I20260812 00:34:21.834332 15922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:21.814660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39451 (local address 127.7.244.65:46865)
0812 00:34:21.815556 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:21.815619 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:21.817609 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:21.818386 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:21.818395 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:21.818970 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:21.819619 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:21.819641 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:21.820755 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:21.820759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:21.822136 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:21.822152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:21.823005 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:21.823013 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:21.823288 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:21.824046 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:21.824079 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:21.826827 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:21.830120 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:21.830129 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:21.830145 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:21.830640 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:21.831048 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:21.831053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:21.831057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:21.831269 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:21.831822 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:21.831831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:21.832217 (+ 386us) client_negotiation.cc:769] Sending connection context
0812 00:34:21.832482 (+ 265us) client_negotiation.cc:240] Negotiation successful
0812 00:34:21.833079 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":340,"threads_started":1}
I20260812 00:34:21.836046 15914 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39451
I20260812 00:34:21.836361 15914 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:34:21.837152 15914 heartbeater.cc:511] Master 127.7.244.126:39451 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
I20260812 00:34:21.839560 15500 ts_manager.cc:194] Registered new tserver with Master: 746dbe1f5c434cd29182b253fe9bd1a1 (127.7.244.65:41047)
I20260812 00:34:21.841121 15500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:46865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:34:21.869127 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39451
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:22.217844 15930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:22.218168 15930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:22.218276 15930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:22.225914 15930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:22.226047 15930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:22.226166 15930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:22.236696 15930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:22.238121 15930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:22.239523 15930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:22.253760 15945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:22.253784 15950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:22.254410 15943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:22.254709 15930 server_base.cc:1034] running on GCE node
I20260812 00:34:22.255401 15930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:22.256526 15930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:22.257738 15930 hybrid_clock.cc:648] HybridClock initialized: now 1786494862257687 us; error 42 us; skew 500 ppm
I20260812 00:34:22.258061 15930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:22 dist-test-slave-gjw7 krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:22.261430 15930 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:34:22.263036 15930 webserver.cc:460] Webserver started at http://127.7.244.66:35727/ using document root <none> and password file <none>
I20260812 00:34:22.263851 15930 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:22.263988 15930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:22.264307 15930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:22.267455 15930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "02078ff5de69453d9c87ea90f6785222"
format_stamp: "Formatted at 2026-08-12 00:34:22 on dist-test-slave-gjw7"
server_key: "7a91e77e99cdfcd508ecff19f42cbe33"
server_key_iv: "3a38bd8b6a8672db79c79b9a524082bc"
server_key_version: "encryptionkey@0"
I20260812 00:34:22.268277 15930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "02078ff5de69453d9c87ea90f6785222"
format_stamp: "Formatted at 2026-08-12 00:34:22 on dist-test-slave-gjw7"
server_key: "7a91e77e99cdfcd508ecff19f42cbe33"
server_key_iv: "3a38bd8b6a8672db79c79b9a524082bc"
server_key_version: "encryptionkey@0"
I20260812 00:34:22.273980 15930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 00:34:22.277861 15963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:22.279311 15930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:34:22.279510 15930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "02078ff5de69453d9c87ea90f6785222"
format_stamp: "Formatted at 2026-08-12 00:34:22 on dist-test-slave-gjw7"
server_key: "7a91e77e99cdfcd508ecff19f42cbe33"
server_key_iv: "3a38bd8b6a8672db79c79b9a524082bc"
server_key_version: "encryptionkey@0"
I20260812 00:34:22.279728 15930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:22.298740 15930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:22.303582 15930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:22.303876 15930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:22.304994 15930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:22.306543 15930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:22.306633 15930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:22.306771 15930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:22.306828 15930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:22.342446 15930 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:36715
I20260812 00:34:22.342504 16223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:36715 every 8 connection(s)
I20260812 00:34:22.343811 15930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:22.345284 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 15930
I20260812 00:34:22.345541 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:22.345970 8145 external_mini_cluster.cc:1428] Setting key 50bbcd54b3e7d6ff22c6d533de069419
Aug 12 00:34:22 dist-test-slave-gjw7 krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:22.367336 16232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:22.346628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:33045 (local address 127.7.244.126:39451)
0812 00:34:22.347131 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:22.347137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:22.348230 (+ 1093us) server_negotiation.cc:407] Connection header received
0812 00:34:22.350505 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:22.350512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:22.350733 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:22.350862 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:22.352626 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.353438 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.355191 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.355678 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.355785 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:22.359989 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:22.359998 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:22.360003 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:22.360053 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:22.363592 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:22.364550 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:22.364556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:22.364559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:22.364753 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:22.365261 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:22.365267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:22.365270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:22.365686 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:22.365868 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:22.366658 (+ 790us) server_negotiation.cc:299] Negotiation successful
0812 00:34:22.366916 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":255,"threads_started":1}
I20260812 00:34:22.368300 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:22.347183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39451 (local address 127.7.244.66:33045)
0812 00:34:22.348062 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:22.348113 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:22.350152 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:22.351180 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:22.351189 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:22.351779 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:22.352420 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:22.352441 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.353612 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.353616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:22.354960 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:22.354981 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.355940 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.355949 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:22.356268 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:22.357060 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:22.357098 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:22.359670 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:22.363805 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:22.363815 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:22.363833 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:22.364388 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:22.364909 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:22.364913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:22.364916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:22.365141 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:22.365852 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:22.365860 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:22.366269 (+ 409us) client_negotiation.cc:769] Sending connection context
0812 00:34:22.366537 (+ 268us) client_negotiation.cc:240] Negotiation successful
0812 00:34:22.367095 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":341,"threads_started":1}
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
I20260812 00:34:22.369884 16227 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39451
I20260812 00:34:22.370217 16227 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:22.371026 16227 heartbeater.cc:511] Master 127.7.244.126:39451 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
I20260812 00:34:22.372963 15500 ts_manager.cc:194] Registered new tserver with Master: 02078ff5de69453d9c87ea90f6785222 (127.7.244.66:36715)
I20260812 00:34:22.374316 15500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:33045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:34:22.402014 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39451
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:22.762918 16242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:22.763265 16242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:22.763404 16242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:22.771039 16242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:22.771190 16242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:22.771351 16242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:22.781699 16242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:22.783187 16242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:22.784818 16242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:22.799049 16254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:22.798985 16257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:22.799077 16251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:22.799888 16242 server_base.cc:1034] running on GCE node
I20260812 00:34:22.800545 16242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:22.801668 16242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:22.802884 16242 hybrid_clock.cc:648] HybridClock initialized: now 1786494862802810 us; error 58 us; skew 500 ppm
I20260812 00:34:22.803241 16242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:22 dist-test-slave-gjw7 krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:22.806830 16242 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:34:22.808426 16242 webserver.cc:460] Webserver started at http://127.7.244.67:42461/ using document root <none> and password file <none>
I20260812 00:34:22.809454 16242 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:22.809607 16242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:22.810017 16242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:22.814250 16242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "5d218c0c62944ca88e36c39ec3440fc9"
format_stamp: "Formatted at 2026-08-12 00:34:22 on dist-test-slave-gjw7"
server_key: "e2ebf99fbdb2083b690d28d70c77cb13"
server_key_iv: "54e00fb3b9a21276a15b48b6fe4f2bba"
server_key_version: "encryptionkey@0"
I20260812 00:34:22.815397 16242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "5d218c0c62944ca88e36c39ec3440fc9"
format_stamp: "Formatted at 2026-08-12 00:34:22 on dist-test-slave-gjw7"
server_key: "e2ebf99fbdb2083b690d28d70c77cb13"
server_key_iv: "54e00fb3b9a21276a15b48b6fe4f2bba"
server_key_version: "encryptionkey@0"
I20260812 00:34:22.821274 16242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 00:34:22.825353 16270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:22.826905 16242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260812 00:34:22.827077 16242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "5d218c0c62944ca88e36c39ec3440fc9"
format_stamp: "Formatted at 2026-08-12 00:34:22 on dist-test-slave-gjw7"
server_key: "e2ebf99fbdb2083b690d28d70c77cb13"
server_key_iv: "54e00fb3b9a21276a15b48b6fe4f2bba"
server_key_version: "encryptionkey@0"
I20260812 00:34:22.827281 16242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:22.840690 16242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:22.844336 15914 heartbeater.cc:503] Master 127.7.244.126:39451 was elected leader, sending a full tablet report...
I20260812 00:34:22.845227 16242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:22.845660 16242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:22.846968 16242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:22.848431 16242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:22.848508 16242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:22.848609 16242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:22.848666 16242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:22.881573 16242 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:42499
I20260812 00:34:22.881621 16520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:42499 every 8 connection(s)
I20260812 00:34:22.882977 16242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:22.886834 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 16242
I20260812 00:34:22.887051 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:22.887519 8145 external_mini_cluster.cc:1428] Setting key c8c1d3b597982211432702fd265de139
Aug 12 00:34:22 dist-test-slave-gjw7 krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:22.905628 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:22.885873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:47599 (local address 127.7.244.126:39451)
0812 00:34:22.886352 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:22.886356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:22.887483 (+ 1127us) server_negotiation.cc:407] Connection header received
0812 00:34:22.889825 (+ 2342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:22.889830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:22.889908 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:22.890003 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:22.891951 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.892792 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.894553 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.894964 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.895064 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:22.898882 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:22.898888 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:22.898891 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:22.898930 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:22.901798 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:22.902846 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:22.902853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:22.902857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:22.903031 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:22.903686 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:22.903691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:22.903694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:22.904052 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:22.904183 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:22.905028 (+ 845us) server_negotiation.cc:299] Negotiation successful
0812 00:34:22.905242 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":230,"threads_started":1}
I20260812 00:34:22.906585 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:22.886427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39451 (local address 127.7.244.67:47599)
0812 00:34:22.887291 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:22.887341 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:22.889525 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:22.890322 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:22.890336 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:22.891118 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:22.891765 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:22.891786 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.892979 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.892984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:22.894411 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:22.894432 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.895216 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.895222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:22.895483 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:22.896188 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:22.896213 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:22.898612 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:22.901983 (+ 3371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:22.901992 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:22.902009 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:22.902687 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:22.903196 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:22.903202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:22.903205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:22.903553 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:22.904174 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:22.904181 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:22.904578 (+ 397us) client_negotiation.cc:769] Sending connection context
0812 00:34:22.904839 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 00:34:22.905338 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":366,"threads_started":1}
I20260812 00:34:22.908324 16521 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39451
I20260812 00:34:22.908751 16521 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:22.909657 16521 heartbeater.cc:511] Master 127.7.244.126:39451 requested a full tablet report, sending...
I20260812 00:34:22.911369 15500 ts_manager.cc:194] Registered new tserver with Master: 5d218c0c62944ca88e36c39ec3440fc9 (127.7.244.67:42499)
I20260812 00:34:22.912225 15500 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:47599
I20260812 00:34:22.916600 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 00:34:22 dist-test-slave-gjw7 krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:34:22 dist-test-slave-gjw7 krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:34:22.973174 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:22.956371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50900 (local address 127.7.244.65:41047)
0812 00:34:22.957073 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:22.957084 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:22.957170 (+ 86us) server_negotiation.cc:407] Connection header received
0812 00:34:22.957431 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:22.957440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:22.957677 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:22.957835 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:22.959489 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.960582 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.962023 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.962645 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.962781 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:22.967284 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:22.967304 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:22.967322 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:22.967366 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:22.970273 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:22.971092 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:22.971102 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:22.971105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:22.971278 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:22.971621 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:22.971624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:22.971627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:22.972062 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:22.972338 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:22.972584 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 00:34:22.972842 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":347,"threads_started":1}
Aug 12 00:34:22 dist-test-slave-gjw7 krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:22.991866 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:22.977841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33918 (local address 127.7.244.126:39451)
0812 00:34:22.978087 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:22.978093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:22.978262 (+ 169us) server_negotiation.cc:407] Connection header received
0812 00:34:22.978420 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:22.978424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:22.978492 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:22.978575 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:22.979864 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.980617 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.981972 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.982408 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.982512 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:22.986746 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:22.986753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:22.986756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:22.986800 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:22.989233 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:22.990025 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:22.990030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:22.990033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:22.990189 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:22.990582 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:22.990586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:22.990589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:22.990970 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:22.991174 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:22.991430 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 00:34:22.991628 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260812 00:34:22.992477 15500 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33918
I20260812 00:34:22.993556 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 15636
I20260812 00:34:23.006778 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:22.994454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50910 (local address 127.7.244.65:41047)
0812 00:34:22.994609 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:22.994614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:22.994947 (+ 333us) server_negotiation.cc:407] Connection header received
0812 00:34:22.995107 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:22.995111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:22.995179 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:22.995255 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:22.996507 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.997213 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.998499 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:22.998938 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:22.999038 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:23.001436 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:23.001443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:23.001447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:23.001489 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:23.004283 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:23.005047 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:23.005051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:23.005053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:23.005193 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:23.005589 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:23.005592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:23.005594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:23.005874 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:23.006078 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:23.006325 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 00:34:23.006526 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 00:34:23.007531 15903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:23.091966 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 15636
I20260812 00:34:23.122341 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 15930
Aug 12 00:34:23 dist-test-slave-gjw7 krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:34:23.140093 16538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:23.123364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.7.244.66:36715)
0812 00:34:23.123935 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:23.123943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:23.123971 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:34:23.124036 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:23.124042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:23.124272 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:23.124412 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:23.126017 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:23.127101 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:23.128517 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:23.129108 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:23.129214 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:23.134089 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:23.134108 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:23.134128 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:23.134172 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:23.137238 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:23.138117 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:23.138125 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:23.138128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:23.138278 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:23.138600 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:23.138603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:23.138606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:23.138995 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:23.139258 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:23.139470 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 00:34:23.139694 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":284,"threads_started":1}
I20260812 00:34:23.141506 16210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:23.226061 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 15930
I20260812 00:34:23.268746 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 16242
Aug 12 00:34:23 dist-test-slave-gjw7 krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:34:23.286545 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:23.269786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.7.244.67:42499)
0812 00:34:23.270436 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:23.270444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:23.270481 (+ 37us) server_negotiation.cc:407] Connection header received
0812 00:34:23.270547 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:23.270554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:23.270776 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:23.270917 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:23.272224 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:23.273426 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:23.274855 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:23.275471 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:23.275610 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:23.280196 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:23.280213 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:23.280231 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:23.280281 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:23.283290 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:23.284085 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:23.284094 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:23.284096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:23.284286 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:23.284719 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:23.284722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:23.284724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:23.285129 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:23.285464 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:23.285902 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 00:34:23.286149 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":314,"threads_started":1}
I20260812 00:34:23.287930 16516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:23.372433 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 16242
I20260812 00:34:23.405275 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 15434
I20260812 00:34:23.406714 15567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:23.511854 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 15434
2026-08-12T00:34:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4357 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:34:23 dist-test-slave-gjw7 krb5kdc[16548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:34:23 dist-test-slave-gjw7 krb5kdc[16548](info): set up 2 sockets
Aug 12 00:34:23 dist-test-slave-gjw7 krb5kdc[16548](info): commencing operation
krb5kdc: starting...
W20260812 00:34:24.820513 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:24 dist-test-slave-gjw7 krb5kdc[16548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:34:25.062325 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:35045
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:45997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:35045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:25.416349 16575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:25.416703 16575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:25.416848 16575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:25.424309 16575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:25.424463 16575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:25.424535 16575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:25.424614 16575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:25.424685 16575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:25.435151 16575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:45997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:35045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:35045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:25.436626 16575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:25.438098 16575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:25.450173 16589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:25.450243 16593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:25.450266 16590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:25.451081 16575 server_base.cc:1034] running on GCE node
I20260812 00:34:25.451956 16575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:25.453536 16575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:25.454800 16575 hybrid_clock.cc:648] HybridClock initialized: now 1786494865454752 us; error 44 us; skew 500 ppm
I20260812 00:34:25.455205 16575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:25 dist-test-slave-gjw7 krb5kdc[16548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494865, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:25.462814 16575 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:34:25.464170 16575 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:25.464287 16575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:25.464603 16575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:25.467880 16575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "1b61d640139c4b81af9b926cdf4fc822"
format_stamp: "Formatted at 2026-08-12 00:34:25 on dist-test-slave-gjw7"
server_key: "1123bcaad14c7dafb89da0566653a8c6"
server_key_iv: "eb62381eff1a9fc733ee6e52d8f93bd1"
server_key_version: "encryptionkey@0"
I20260812 00:34:25.468717 16575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "1b61d640139c4b81af9b926cdf4fc822"
format_stamp: "Formatted at 2026-08-12 00:34:25 on dist-test-slave-gjw7"
server_key: "1123bcaad14c7dafb89da0566653a8c6"
server_key_iv: "eb62381eff1a9fc733ee6e52d8f93bd1"
server_key_version: "encryptionkey@0"
I20260812 00:34:25.475809 16575 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260812 00:34:25.482554 16603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:25.484566 16575 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260812 00:34:25.484759 16575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "1b61d640139c4b81af9b926cdf4fc822"
format_stamp: "Formatted at 2026-08-12 00:34:25 on dist-test-slave-gjw7"
server_key: "1123bcaad14c7dafb89da0566653a8c6"
server_key_iv: "eb62381eff1a9fc733ee6e52d8f93bd1"
server_key_version: "encryptionkey@0"
I20260812 00:34:25.485045 16575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:25.506268 16575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:25.512652 16575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:25.513024 16575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:25.535596 16575 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:35045
I20260812 00:34:25.535614 16698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:35045 every 8 connection(s)
I20260812 00:34:25.536674 16575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:25.540772 16700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:25.546959 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 16575
I20260812 00:34:25.547200 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:25.547880 8145 external_mini_cluster.cc:1428] Setting key 3b099680fb66578592b78a7c4c7982ec
I20260812 00:34:25.553712 16700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822: Bootstrap starting.
I20260812 00:34:25.557869 16700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:25.559288 16700 log.cc:824] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:25.563351 16700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822: No bootstrap required, opened a new log
Aug 12 00:34:25 dist-test-slave-gjw7 krb5kdc[16548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:25.571990 16700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61d640139c4b81af9b926cdf4fc822" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 35045 } }
I20260812 00:34:25.572441 16700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:25.572556 16700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b61d640139c4b81af9b926cdf4fc822, State: Initialized, Role: FOLLOWER
I20260812 00:34:25.573307 16700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61d640139c4b81af9b926cdf4fc822" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 35045 } }
I20260812 00:34:25.573616 16700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:25.573770 16700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:25.573963 16700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:25.575139 16706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:25.550260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60862 (local address 127.7.244.126:35045)
0812 00:34:25.551198 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:25.551216 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:25.551271 (+ 55us) server_negotiation.cc:407] Connection header received
0812 00:34:25.553779 (+ 2508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:25.553808 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:25.554288 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:25.554644 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:25.556697 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:25.558233 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:25.560253 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:25.561237 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:25.561442 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:25.566404 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:25.566419 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:25.566438 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:25.566494 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:25.570023 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:25.570881 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:25.570889 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:25.570897 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:25.571115 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:25.571589 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:25.571593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:25.571595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:25.572231 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:25.572548 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:25.573068 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 00:34:25.573576 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":375,"threads_started":1}
I20260812 00:34:25.577014 16700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61d640139c4b81af9b926cdf4fc822" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 35045 } }
I20260812 00:34:25.577687 16700 leader_election.cc:302] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b61d640139c4b81af9b926cdf4fc822; no voters:
I20260812 00:34:25.578742 16700 leader_election.cc:288] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:25.578819 16708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:25.579489 16708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [term 1 LEADER]: Becoming Leader. State: Replica: 1b61d640139c4b81af9b926cdf4fc822, State: Running, Role: LEADER
I20260812 00:34:25.580066 16708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61d640139c4b81af9b926cdf4fc822" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 35045 } }
I20260812 00:34:25.580878 16700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:25.582949 16711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b61d640139c4b81af9b926cdf4fc822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61d640139c4b81af9b926cdf4fc822" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 35045 } } }
I20260812 00:34:25.583349 16711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:25.585058 16717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:25.585012 16712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b61d640139c4b81af9b926cdf4fc822. Latest consensus state: current_term: 1 leader_uuid: "1b61d640139c4b81af9b926cdf4fc822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b61d640139c4b81af9b926cdf4fc822" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 35045 } } }
I20260812 00:34:25.585441 16712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:25.591370 16717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:25.602051 16717 catalog_manager.cc:1347] Generated new cluster ID: ee45cf7e504f4778a16a045da023a1b7
I20260812 00:34:25.602190 16717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:25.619038 16717 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:25.620450 16717 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:25.639032 16717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822: Generated new TSK 0
I20260812 00:34:25.639967 16717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:34:25.710497 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:35045
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:45997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:26.085132 16743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:26.085558 16743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:26.085690 16743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:26.093547 16743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:26.093690 16743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:26.093853 16743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:26.104384 16743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:45997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:35045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:26.105952 16743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:26.107465 16743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:26.121544 16753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:26.121575 16754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:26.121619 16757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:26.122460 16743 server_base.cc:1034] running on GCE node
I20260812 00:34:26.123157 16743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:26.124222 16743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:26.125404 16743 hybrid_clock.cc:648] HybridClock initialized: now 1786494866125332 us; error 58 us; skew 500 ppm
I20260812 00:34:26.125758 16743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:26 dist-test-slave-gjw7 krb5kdc[16548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:26.133252 16743 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:34:26.134785 16743 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:26.134914 16743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:26.135315 16743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:26.138684 16743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "a254309ab1f1421387c02895f35a6946"
format_stamp: "Formatted at 2026-08-12 00:34:26 on dist-test-slave-gjw7"
server_key: "d19d21d77ae644ce0ee668732f0b4a3e"
server_key_iv: "c0fa42298e6dd6481faf228e0e4e070a"
server_key_version: "encryptionkey@0"
I20260812 00:34:26.139635 16743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "a254309ab1f1421387c02895f35a6946"
format_stamp: "Formatted at 2026-08-12 00:34:26 on dist-test-slave-gjw7"
server_key: "d19d21d77ae644ce0ee668732f0b4a3e"
server_key_iv: "c0fa42298e6dd6481faf228e0e4e070a"
server_key_version: "encryptionkey@0"
I20260812 00:34:26.145638 16743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 00:34:26.149777 16769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:26.151367 16743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:34:26.151569 16743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "a254309ab1f1421387c02895f35a6946"
format_stamp: "Formatted at 2026-08-12 00:34:26 on dist-test-slave-gjw7"
server_key: "d19d21d77ae644ce0ee668732f0b4a3e"
server_key_iv: "c0fa42298e6dd6481faf228e0e4e070a"
server_key_version: "encryptionkey@0"
I20260812 00:34:26.151844 16743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:26.166698 16743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:26.171633 16743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:26.171944 16743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:26.173030 16743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:26.174552 16743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:26.174649 16743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:26.174741 16743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:26.174815 16743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:26.207549 16743 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:44289
I20260812 00:34:26.207631 17010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:44289 every 8 connection(s)
I20260812 00:34:26.208448 16743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:26.208811 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 16743
I20260812 00:34:26.209009 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:26.209882 8145 external_mini_cluster.cc:1428] Setting key fbb70bfd50cc6ee424cc425905216014
Aug 12 00:34:26 dist-test-slave-gjw7 krb5kdc[16548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:26.230993 17013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:26.211130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:38281 (local address 127.7.244.126:35045)
0812 00:34:26.211599 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:26.211605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:26.212874 (+ 1269us) server_negotiation.cc:407] Connection header received
0812 00:34:26.214810 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:26.214816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:26.214899 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:26.215012 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:26.216620 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:26.217631 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:26.219249 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:26.219702 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:26.219804 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:26.224237 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:26.224244 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:26.224247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:26.224287 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:26.227024 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:26.228106 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:26.228111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:26.228113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:26.228296 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:26.228906 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:26.228911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:26.228914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:26.229249 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:26.229447 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:26.230500 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0812 00:34:26.230713 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":222,"threads_started":1}
I20260812 00:34:26.232801 17014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:26.211759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:35045 (local address 127.7.244.65:38281)
0812 00:34:26.212675 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:26.212727 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:26.214527 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:26.215226 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:26.215235 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:26.215796 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:26.216420 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:26.216440 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:26.217798 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:26.217803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:26.219082 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:26.219095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:26.219956 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:26.219963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:26.220243 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:26.221018 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:26.221052 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:26.223936 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:26.227230 (+ 3294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:26.227242 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:26.227263 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:26.227958 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:26.228435 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:26.228441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:26.228444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:26.228743 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:26.229452 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:26.229463 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:26.230006 (+ 543us) client_negotiation.cc:769] Sending connection context
0812 00:34:26.230382 (+ 376us) client_negotiation.cc:240] Negotiation successful
0812 00:34:26.231031 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":360,"threads_started":1}
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:34:26.234562 17011 heartbeater.cc:347] Connected to a master server at 127.7.244.126:35045
I20260812 00:34:26.234901 17011 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:26.235598 17011 heartbeater.cc:511] Master 127.7.244.126:35045 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
I20260812 00:34:26.238099 16632 ts_manager.cc:194] Registered new tserver with Master: a254309ab1f1421387c02895f35a6946 (127.7.244.65:44289)
I20260812 00:34:26.239656 16632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:38281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:34:26.264717 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:35045
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:45997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:26.611650 17023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:26.611997 17023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:26.612131 17023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:26.619562 17023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:26.619714 17023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:26.619872 17023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:26.630915 17023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:45997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:35045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:26.632373 17023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:26.633792 17023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:26.648373 17034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:26.648485 17040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:26.648458 17033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:26.655331 17023 server_base.cc:1034] running on GCE node
I20260812 00:34:26.656191 17023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:26.657505 17023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:26.658805 17023 hybrid_clock.cc:648] HybridClock initialized: now 1786494866658734 us; error 55 us; skew 500 ppm
I20260812 00:34:26.659241 17023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:26 dist-test-slave-gjw7 krb5kdc[16548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:26.662865 17023 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:34:26.664247 17023 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:26.664374 17023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:26.664741 17023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:26.668268 17023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "d299544d3c1e4a26804c145b76e6c118"
format_stamp: "Formatted at 2026-08-12 00:34:26 on dist-test-slave-gjw7"
server_key: "97f565d1cda0c460a67afe5e541648a5"
server_key_iv: "d4d6eeec8bc429667272040a3a147c50"
server_key_version: "encryptionkey@0"
I20260812 00:34:26.669188 17023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "d299544d3c1e4a26804c145b76e6c118"
format_stamp: "Formatted at 2026-08-12 00:34:26 on dist-test-slave-gjw7"
server_key: "97f565d1cda0c460a67afe5e541648a5"
server_key_iv: "d4d6eeec8bc429667272040a3a147c50"
server_key_version: "encryptionkey@0"
I20260812 00:34:26.674885 17023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 00:34:26.678949 17054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:26.680546 17023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:34:26.680747 17023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "d299544d3c1e4a26804c145b76e6c118"
format_stamp: "Formatted at 2026-08-12 00:34:26 on dist-test-slave-gjw7"
server_key: "97f565d1cda0c460a67afe5e541648a5"
server_key_iv: "d4d6eeec8bc429667272040a3a147c50"
server_key_version: "encryptionkey@0"
I20260812 00:34:26.680994 17023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:26.698814 17023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:26.703114 17023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:26.703392 17023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:26.704421 17023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:26.706004 17023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:26.706100 17023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:26.706215 17023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:26.706308 17023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:26.739320 17023 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:36633
I20260812 00:34:26.739357 17282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:36633 every 8 connection(s)
I20260812 00:34:26.740242 17023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:26.748934 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 17023
I20260812 00:34:26.749140 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:26.749634 8145 external_mini_cluster.cc:1428] Setting key bddf4ffbe78aee4a8c50d4747e3c628f
Aug 12 00:34:26 dist-test-slave-gjw7 krb5kdc[16548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494866, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:26.762857 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:26.742791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:49093 (local address 127.7.244.126:35045)
0812 00:34:26.743322 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:26.743332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:26.744386 (+ 1054us) server_negotiation.cc:407] Connection header received
0812 00:34:26.746461 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:26.746466 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:26.746547 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:26.746667 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:26.748396 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:26.749123 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:26.751084 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:26.751598 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:26.751716 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:26.755715 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:26.755722 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:26.755726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:26.755773 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:26.759072 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:26.760089 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:26.760094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:26.760097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:26.760253 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:26.760829 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:26.760833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:26.760835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:26.761204 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:26.761375 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:26.762344 (+ 969us) server_negotiation.cc:299] Negotiation successful
0812 00:34:26.762554 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":215,"threads_started":1}
I20260812 00:34:26.764354 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:26.743454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:35045 (local address 127.7.244.66:49093)
0812 00:34:26.744192 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:26.744242 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:26.746109 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:26.746901 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:26.746911 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:26.747528 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:26.748177 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:26.748197 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:26.749333 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:26.749339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:26.750920 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:26.750936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:26.751870 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:26.751881 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:26.752107 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:26.752792 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:26.752828 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:26.755385 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:26.759284 (+ 3899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:26.759294 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:26.759312 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:26.759925 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:26.760390 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:26.760394 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:26.760398 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:26.760681 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:26.761407 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:26.761417 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:26.761911 (+ 494us) client_negotiation.cc:769] Sending connection context
0812 00:34:26.762238 (+ 327us) client_negotiation.cc:240] Negotiation successful
0812 00:34:26.762865 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":289,"threads_started":1}
I20260812 00:34:26.766084 17283 heartbeater.cc:347] Connected to a master server at 127.7.244.126:35045
I20260812 00:34:26.766467 17283 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:26.767215 17283 heartbeater.cc:511] Master 127.7.244.126:35045 requested a full tablet report, sending...
I20260812 00:34:26.768891 16632 ts_manager.cc:194] Registered new tserver with Master: d299544d3c1e4a26804c145b76e6c118 (127.7.244.66:36633)
I20260812 00:34:26.769789 16632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:49093
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:34:26.805456 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:35045
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:45997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:27.160025 17296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:27.160386 17296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:27.160537 17296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:27.168025 17296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:27.168169 17296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:27.168334 17296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:27.178493 17296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:45997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:35045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:27.179888 17296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:27.181365 17296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:27.195223 17319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:27.195273 17323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:27.195243 17318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:27.195737 17296 server_base.cc:1034] running on GCE node
I20260812 00:34:27.196573 17296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:27.197808 17296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:27.199052 17296 hybrid_clock.cc:648] HybridClock initialized: now 1786494867198987 us; error 61 us; skew 500 ppm
I20260812 00:34:27.199455 17296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:27 dist-test-slave-gjw7 krb5kdc[16548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494867, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:27.203263 17296 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:34:27.204573 17296 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:27.204696 17296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:27.205049 17296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:27.208408 17296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "5b60bb3edb1342ccb774b3de59ed25be"
format_stamp: "Formatted at 2026-08-12 00:34:27 on dist-test-slave-gjw7"
server_key: "51eb27cf68a3616427a43e5db110088c"
server_key_iv: "9e7379fe063762b51b5bbbde7fb8dc90"
server_key_version: "encryptionkey@0"
I20260812 00:34:27.209318 17296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "5b60bb3edb1342ccb774b3de59ed25be"
format_stamp: "Formatted at 2026-08-12 00:34:27 on dist-test-slave-gjw7"
server_key: "51eb27cf68a3616427a43e5db110088c"
server_key_iv: "9e7379fe063762b51b5bbbde7fb8dc90"
server_key_version: "encryptionkey@0"
I20260812 00:34:27.214795 17296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 00:34:27.218822 17336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:27.220350 17296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:34:27.220531 17296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "5b60bb3edb1342ccb774b3de59ed25be"
format_stamp: "Formatted at 2026-08-12 00:34:27 on dist-test-slave-gjw7"
server_key: "51eb27cf68a3616427a43e5db110088c"
server_key_iv: "9e7379fe063762b51b5bbbde7fb8dc90"
server_key_version: "encryptionkey@0"
I20260812 00:34:27.220775 17296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:27.238828 17296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:27.243139 17011 heartbeater.cc:503] Master 127.7.244.126:35045 was elected leader, sending a full tablet report...
I20260812 00:34:27.243676 17296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:27.243961 17296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:27.245507 17296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:27.247037 17296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:27.247143 17296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:27.247236 17296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:27.247287 17296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:27.280198 17296 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:35257
I20260812 00:34:27.280256 17579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:35257 every 8 connection(s)
I20260812 00:34:27.281184 17296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:27.290244 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 17296
I20260812 00:34:27.290500 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:27.291090 8145 external_mini_cluster.cc:1428] Setting key 7bc10de542894b4e0d8e14779b3a22a6
Aug 12 00:34:27 dist-test-slave-gjw7 krb5kdc[16548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494867, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:27.302793 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.283735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:55837 (local address 127.7.244.126:35045)
0812 00:34:27.284264 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.284269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.285350 (+ 1081us) server_negotiation.cc:407] Connection header received
0812 00:34:27.287398 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.287404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.287542 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.287640 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:27.289643 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.290466 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.292624 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.293056 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.293188 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.296943 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:27.296949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:27.296952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:27.296991 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:27.299747 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.300562 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.300566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.300569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.300698 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.301077 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.301080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.301082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.301409 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:27.301536 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.302298 (+ 762us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.302502 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":240,"threads_started":1}
I20260812 00:34:27.303809 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.284309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:35045 (local address 127.7.244.67:55837)
0812 00:34:27.285143 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:27.285217 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:27.287096 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:27.287914 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:27.287923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:27.288673 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:27.289412 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.289442 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.290650 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.290656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.292500 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.292514 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.293361 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.293370 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.293621 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.294349 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:27.294379 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:27.296709 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:27.299927 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:27.299935 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:27.299951 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:27.300449 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:27.300778 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:27.300781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:27.300783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:27.300992 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:27.301518 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:27.301525 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:27.301886 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 00:34:27.302150 (+ 264us) client_negotiation.cc:240] Negotiation successful
0812 00:34:27.302605 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":312,"threads_started":1}
I20260812 00:34:27.305449 17580 heartbeater.cc:347] Connected to a master server at 127.7.244.126:35045
I20260812 00:34:27.305830 17580 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:27.306617 17580 heartbeater.cc:511] Master 127.7.244.126:35045 requested a full tablet report, sending...
I20260812 00:34:27.308131 16632 ts_manager.cc:194] Registered new tserver with Master: 5b60bb3edb1342ccb774b3de59ed25be (127.7.244.67:35257)
I20260812 00:34:27.308972 16632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:55837
I20260812 00:34:27.319769 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:34:27.338711 17587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.324568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.7.244.126:35045)
0812 00:34:27.324817 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.324823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.325230 (+ 407us) server_negotiation.cc:407] Connection header received
0812 00:34:27.325412 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.325416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.325488 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.325573 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:27.327124 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.327874 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.329250 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.329736 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.329873 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.332568 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:27.332574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:27.332577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:27.332621 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:27.335559 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.336434 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.336439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.336441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.336593 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.337024 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.337028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.337030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.337464 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:27.337716 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.338107 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.338369 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 00:34:27.343312 16632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60872:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:27.350597 16632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:27.376785 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.367116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:36633 (local address 127.0.0.1:47028)
0812 00:34:27.368701 (+ 1585us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:27.368729 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:27.368873 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:27.369640 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:27.369648 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:27.369695 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:27.370088 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.370102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.371689 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.371695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.373771 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.373793 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.375761 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.375768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.376020 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.376070 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 00:34:27.376187 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 00:34:27.376407 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1451,"thread_start_us":1295,"threads_started":1}
I20260812 00:34:27.376789 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.367694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.7.244.66:36633)
0812 00:34:27.368320 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.368330 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.368856 (+ 526us) server_negotiation.cc:407] Connection header received
0812 00:34:27.369064 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.369073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.369394 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.369604 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:27.370320 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.371540 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.373995 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.375554 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.375681 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.375906 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.376278 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.376468 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":311,"threads_started":1}
I20260812 00:34:27.378268 17612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.365397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44289 (local address 127.0.0.1:41060)
0812 00:34:27.367047 (+ 1650us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:27.367074 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:27.367215 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:27.370176 (+ 2961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:27.370182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:27.370213 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:27.370709 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.370722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.373146 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.373152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.375317 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.375335 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.377374 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.377384 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.377601 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.377636 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:34:27.377740 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 00:34:27.377895 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1500,"thread_start_us":1309,"threads_started":1}
I20260812 00:34:27.379419 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.366057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.7.244.65:44289)
0812 00:34:27.369513 (+ 3456us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.369522 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.369551 (+ 29us) server_negotiation.cc:407] Connection header received
0812 00:34:27.369673 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.369683 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.369928 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.370942 (+ 1014us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:27.371201 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.372965 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.375533 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.377086 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.377234 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.377509 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.377837 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.379089 (+ 1252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3285,"thread_start_us":3078,"threads_started":1}
I20260812 00:34:27.381637 17607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.365025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:35257 (local address 127.0.0.1:50418)
0812 00:34:27.366016 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:27.366080 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:27.366276 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:27.374402 (+ 8126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:27.374406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:27.374433 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:27.374884 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.374898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.377135 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.377140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.379223 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.379240 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.380724 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.380731 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.380965 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.381000 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:34:27.381102 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 00:34:27.381245 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":1,"thread_start_us":562,"threads_started":1}
I20260812 00:34:27.381752 17611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.365263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50418 (local address 127.7.244.67:35257)
0812 00:34:27.373684 (+ 8421us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.373692 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.373723 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:34:27.373832 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.373840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.374146 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.375075 (+ 929us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:27.375328 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.376954 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.379476 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.380487 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.380614 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.380861 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.381171 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.381389 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8262,"thread_start_us":8044,"threads_started":1}
I20260812 00:34:27.381563 17166 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd7e64a3885425dbad510884d4c61cf (DEFAULT_TABLE table=test-table [id=dfbd8a85aa0947bc99db1d27cdf705e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:27.383136 17166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd7e64a3885425dbad510884d4c61cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:27.384624 16878 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd7e64a3885425dbad510884d4c61cf (DEFAULT_TABLE table=test-table [id=dfbd8a85aa0947bc99db1d27cdf705e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:27.386487 16878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd7e64a3885425dbad510884d4c61cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:27.386032 17436 tablet_service.cc:1467] Processing CreateTablet for tablet 3bd7e64a3885425dbad510884d4c61cf (DEFAULT_TABLE table=test-table [id=dfbd8a85aa0947bc99db1d27cdf705e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:27.387606 17436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bd7e64a3885425dbad510884d4c61cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:27.400825 17622 tablet_bootstrap.cc:492] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Bootstrap starting.
I20260812 00:34:27.401492 17623 tablet_bootstrap.cc:492] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Bootstrap starting.
I20260812 00:34:27.403622 17626 tablet_bootstrap.cc:492] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Bootstrap starting.
I20260812 00:34:27.405071 17623 tablet_bootstrap.cc:654] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:27.405071 17622 tablet_bootstrap.cc:654] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:27.406595 17623 log.cc:824] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:27.406595 17622 log.cc:824] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:27.407132 17626 tablet_bootstrap.cc:654] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:27.408596 17626 log.cc:824] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:27.410310 17623 tablet_bootstrap.cc:492] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: No bootstrap required, opened a new log
I20260812 00:34:27.410322 17622 tablet_bootstrap.cc:492] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: No bootstrap required, opened a new log
I20260812 00:34:27.410629 17622 ts_tablet_manager.cc:1397] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 00:34:27.410629 17623 ts_tablet_manager.cc:1397] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260812 00:34:27.415859 17626 tablet_bootstrap.cc:492] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: No bootstrap required, opened a new log
I20260812 00:34:27.416178 17626 ts_tablet_manager.cc:1397] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260812 00:34:27.422168 17623 raft_consensus.cc:348] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.422179 17622 raft_consensus.cc:348] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.422785 17623 raft_consensus.cc:374] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:27.422784 17622 raft_consensus.cc:374] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:27.422916 17622 raft_consensus.cc:729] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a254309ab1f1421387c02895f35a6946, State: Initialized, Role: FOLLOWER
I20260812 00:34:27.422969 17623 raft_consensus.cc:729] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d299544d3c1e4a26804c145b76e6c118, State: Initialized, Role: FOLLOWER
I20260812 00:34:27.423732 17622 consensus_queue.cc:260] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.423734 17623 consensus_queue.cc:260] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.424911 17626 raft_consensus.cc:348] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.425562 17626 raft_consensus.cc:374] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:27.425727 17626 raft_consensus.cc:729] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b60bb3edb1342ccb774b3de59ed25be, State: Initialized, Role: FOLLOWER
I20260812 00:34:27.426509 17622 ts_tablet_manager.cc:1428] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260812 00:34:27.426594 17626 consensus_queue.cc:260] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.427803 17283 heartbeater.cc:503] Master 127.7.244.126:35045 was elected leader, sending a full tablet report...
I20260812 00:34:27.427889 17580 heartbeater.cc:503] Master 127.7.244.126:35045 was elected leader, sending a full tablet report...
I20260812 00:34:27.429025 17623 ts_tablet_manager.cc:1428] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Time spent starting tablet: real 0.018s user 0.014s sys 0.003s
I20260812 00:34:27.430498 17626 ts_tablet_manager.cc:1428] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Time spent starting tablet: real 0.014s user 0.008s sys 0.005s
W20260812 00:34:27.461025 17012 tablet.cc:2369] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:27.468330 17638 raft_consensus.cc:482] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:27.468734 17638 raft_consensus.cc:504] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.470851 17638 leader_election.cc:288] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b60bb3edb1342ccb774b3de59ed25be (127.7.244.67:35257), d299544d3c1e4a26804c145b76e6c118 (127.7.244.66:36633)
I20260812 00:34:27.479139 17611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.471214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:38411 (local address 127.7.244.67:35257)
0812 00:34:27.471546 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.471553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.472478 (+ 925us) server_negotiation.cc:407] Connection header received
0812 00:34:27.472640 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.472646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.472726 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.472851 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:27.473537 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.474260 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.476411 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.477773 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.477940 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.478036 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.478691 (+ 655us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.478873 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 00:34:27.479310 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.471210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:35257 (local address 127.7.244.65:38411)
0812 00:34:27.472354 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:27.472371 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:27.472515 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:27.472885 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:27.472889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:27.472914 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:27.473364 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.473379 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.474411 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.474415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.476196 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.476215 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.478266 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.478273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.478482 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.478524 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 00:34:27.478645 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 00:34:27.478828 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"mutex_wait_us":31,"thread_start_us":808,"threads_started":1}
I20260812 00:34:27.480552 17482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd7e64a3885425dbad510884d4c61cf" candidate_uuid: "a254309ab1f1421387c02895f35a6946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b60bb3edb1342ccb774b3de59ed25be" is_pre_election: true
I20260812 00:34:27.481405 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.471575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:58573 (local address 127.7.244.66:36633)
0812 00:34:27.471833 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.471840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.472285 (+ 445us) server_negotiation.cc:407] Connection header received
0812 00:34:27.472496 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.472504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.472588 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.472682 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:27.473313 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.474134 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.476298 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.480105 (+ 3807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.480302 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.480393 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.480769 (+ 376us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.481077 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260812 00:34:27.481356 17482 raft_consensus.cc:2454] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a254309ab1f1421387c02895f35a6946 in term 0.
I20260812 00:34:27.481437 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.471194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:36633 (local address 127.7.244.65:58573)
0812 00:34:27.472142 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:27.472166 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:27.472348 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:27.472702 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:27.472707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:27.472732 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:27.473133 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.473146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.474298 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.474303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.476068 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:27.476089 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.480355 (+ 4266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.480363 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:27.480552 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.480588 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 00:34:27.480698 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 00:34:27.480848 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":40,"thread_start_us":558,"threads_started":1}
I20260812 00:34:27.482316 16770 leader_election.cc:302] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b60bb3edb1342ccb774b3de59ed25be, a254309ab1f1421387c02895f35a6946; no voters:
I20260812 00:34:27.482877 17638 raft_consensus.cc:2781] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:27.483048 17638 raft_consensus.cc:482] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:27.483158 17638 raft_consensus.cc:3037] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:27.483911 17202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd7e64a3885425dbad510884d4c61cf" candidate_uuid: "a254309ab1f1421387c02895f35a6946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d299544d3c1e4a26804c145b76e6c118" is_pre_election: true
I20260812 00:34:27.484504 17202 raft_consensus.cc:2454] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a254309ab1f1421387c02895f35a6946 in term 0.
I20260812 00:34:27.486286 17638 raft_consensus.cc:504] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.487076 17638 leader_election.cc:288] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [CANDIDATE]: Term 1 election: Requested vote from peers 5b60bb3edb1342ccb774b3de59ed25be (127.7.244.67:35257), d299544d3c1e4a26804c145b76e6c118 (127.7.244.66:36633)
I20260812 00:34:27.487679 17482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd7e64a3885425dbad510884d4c61cf" candidate_uuid: "a254309ab1f1421387c02895f35a6946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b60bb3edb1342ccb774b3de59ed25be"
I20260812 00:34:27.487934 17482 raft_consensus.cc:3037] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:27.488281 17202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bd7e64a3885425dbad510884d4c61cf" candidate_uuid: "a254309ab1f1421387c02895f35a6946" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d299544d3c1e4a26804c145b76e6c118"
I20260812 00:34:27.488538 17202 raft_consensus.cc:3037] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:27.490617 17482 raft_consensus.cc:2454] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a254309ab1f1421387c02895f35a6946 in term 1.
I20260812 00:34:27.491111 16770 leader_election.cc:302] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b60bb3edb1342ccb774b3de59ed25be, a254309ab1f1421387c02895f35a6946; no voters:
I20260812 00:34:27.491245 17202 raft_consensus.cc:2454] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a254309ab1f1421387c02895f35a6946 in term 1.
I20260812 00:34:27.491461 17638 raft_consensus.cc:2781] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:27.491988 17638 raft_consensus.cc:686] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 1 LEADER]: Becoming Leader. State: Replica: a254309ab1f1421387c02895f35a6946, State: Running, Role: LEADER
W20260812 00:34:27.492161 17287 tablet.cc:2369] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:27.492938 17638 consensus_queue.cc:237] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } }
I20260812 00:34:27.498317 16630 catalog_manager.cc:5495] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 reported cstate change: term changed from 0 to 1, leader changed from <none> to a254309ab1f1421387c02895f35a6946 (127.7.244.65). New cstate: current_term: 1 leader_uuid: "a254309ab1f1421387c02895f35a6946" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a254309ab1f1421387c02895f35a6946" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 } health_report { overall_health: UNKNOWN } } }
W20260812 00:34:27.532724 17582 tablet.cc:2369] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:27.538645 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.532335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.7.244.65:44289)
0812 00:34:27.532546 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.532550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.532570 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:34:27.532806 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.532811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.532910 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.533080 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:27.533619 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.534371 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.536196 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.536631 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.536754 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.537373 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:27.537968 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:27.538100 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.538276 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.538374 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 00:34:27.548780 17202 raft_consensus.cc:1261] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 1 FOLLOWER]: Refusing update from remote peer a254309ab1f1421387c02895f35a6946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:27.548821 17482 raft_consensus.cc:1261] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 1 FOLLOWER]: Refusing update from remote peer a254309ab1f1421387c02895f35a6946: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:27.549929 17646 consensus_queue.cc:1035] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b60bb3edb1342ccb774b3de59ed25be" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 35257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:27.550302 17638 consensus_queue.cc:1035] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d299544d3c1e4a26804c145b76e6c118" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:27.560611 17652 mvcc.cc:205] Tried to move back new op lower bound from 7317482977468227584 to 7317482977255849984. Current Snapshot: MvccSnapshot[applied={T|T < 7317482977468227584}]
I20260812 00:34:27.562575 17656 mvcc.cc:205] Tried to move back new op lower bound from 7317482977468227584 to 7317482977255849984. Current Snapshot: MvccSnapshot[applied={T|T < 7317482977468227584}]
I20260812 00:34:27.564113 17655 mvcc.cc:205] Tried to move back new op lower bound from 7317482977468227584 to 7317482977255849984. Current Snapshot: MvccSnapshot[applied={T|T < 7317482977468227584}]
I20260812 00:34:27.576948 16630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:27.577322 16630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60872:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:27.581719 16630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b61d640139c4b81af9b926cdf4fc822: Sending DeleteTablet for 3 replicas of tablet 3bd7e64a3885425dbad510884d4c61cf
I20260812 00:34:27.583118 17436 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd7e64a3885425dbad510884d4c61cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:27 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:50418
I20260812 00:34:27.583179 16878 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd7e64a3885425dbad510884d4c61cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:27 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:41060
I20260812 00:34:27.583559 17166 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bd7e64a3885425dbad510884d4c61cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:27 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:47028
I20260812 00:34:27.586025 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 16743
I20260812 00:34:27.596549 17673 tablet_replica.cc:330] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: stopping tablet replica
I20260812 00:34:27.597122 17673 raft_consensus.cc:2229] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:27.597800 17673 raft_consensus.cc:2258] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:27.598860 17672 tablet_replica.cc:330] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: stopping tablet replica
I20260812 00:34:27.599376 17672 raft_consensus.cc:2229] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:27.600122 17672 raft_consensus.cc:2258] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:27.601567 17673 ts_tablet_manager.cc:1905] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:27.602216 17671 tablet_replica.cc:330] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: stopping tablet replica
I20260812 00:34:27.602304 17672 ts_tablet_manager.cc:1905] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:27.602813 17671 raft_consensus.cc:2229] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:27.603428 17671 raft_consensus.cc:2258] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 12 00:34:27 dist-test-slave-gjw7 krb5kdc[16548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:34:27.610193 17672 ts_tablet_manager.cc:1918] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:27.610394 17672 log.cc:1196] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/3bd7e64a3885425dbad510884d4c61cf
I20260812 00:34:27.611027 17672 ts_tablet_manager.cc:1939] T 3bd7e64a3885425dbad510884d4c61cf P a254309ab1f1421387c02895f35a6946: Deleting consensus metadata
I20260812 00:34:27.611469 17673 ts_tablet_manager.cc:1918] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:27.611757 17673 log.cc:1196] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/3bd7e64a3885425dbad510884d4c61cf
I20260812 00:34:27.611876 17671 ts_tablet_manager.cc:1905] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:27.612522 17673 ts_tablet_manager.cc:1939] T 3bd7e64a3885425dbad510884d4c61cf P d299544d3c1e4a26804c145b76e6c118: Deleting consensus metadata
I20260812 00:34:27.613145 16607 catalog_manager.cc:4841] TS a254309ab1f1421387c02895f35a6946 (127.7.244.65:44289): tablet 3bd7e64a3885425dbad510884d4c61cf (table test-table [id=dfbd8a85aa0947bc99db1d27cdf705e9]) successfully deleted
I20260812 00:34:27.615520 16607 catalog_manager.cc:4841] TS d299544d3c1e4a26804c145b76e6c118 (127.7.244.66:36633): tablet 3bd7e64a3885425dbad510884d4c61cf (table test-table [id=dfbd8a85aa0947bc99db1d27cdf705e9]) successfully deleted
I20260812 00:34:27.618676 17615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.587791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.7.244.65:44289)
0812 00:34:27.588017 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.588025 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.598982 (+ 10957us) server_negotiation.cc:407] Connection header received
0812 00:34:27.599267 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.599273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.599374 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.599474 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:27.601361 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.602425 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.604263 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.604993 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.605099 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.610747 (+ 5648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:27.610764 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:27.610795 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:27.610856 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:27.615203 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.616292 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.616306 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.616309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.616506 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.616932 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.616938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.616942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.617407 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:27.617728 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.618024 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.618297 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 00:34:27.619412 17005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:27.619405 17671 ts_tablet_manager.cc:1918] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:27.619596 17671 log.cc:1196] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/3bd7e64a3885425dbad510884d4c61cf
I20260812 00:34:27.620067 17671 ts_tablet_manager.cc:1939] T 3bd7e64a3885425dbad510884d4c61cf P 5b60bb3edb1342ccb774b3de59ed25be: Deleting consensus metadata
I20260812 00:34:27.622001 16605 catalog_manager.cc:4841] TS 5b60bb3edb1342ccb774b3de59ed25be (127.7.244.67:35257): tablet 3bd7e64a3885425dbad510884d4c61cf (table test-table [id=dfbd8a85aa0947bc99db1d27cdf705e9]) successfully deleted
I20260812 00:34:27.719611 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 16743
I20260812 00:34:27.751761 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 17023
Aug 12 00:34:27 dist-test-slave-gjw7 krb5kdc[16548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:34:27.769706 17620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.752627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.7.244.66:36633)
0812 00:34:27.752802 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.752806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.752824 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:34:27.752969 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.752973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.753036 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.753117 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:27.754823 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.755623 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.757515 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.758004 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.758120 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.763254 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:27.763269 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:27.763286 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:27.763348 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:27.766446 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.767312 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.767321 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.767323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.767479 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.767931 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.767935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.767937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.768404 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:27.768702 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.769068 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.769357 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 00:34:27.770552 17277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:27.863152 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 17023
I20260812 00:34:27.897917 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 17296
Aug 12 00:34:27 dist-test-slave-gjw7 krb5kdc[16548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494864, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:34:27.913512 17611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:27.898724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.7.244.67:35257)
0812 00:34:27.898927 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:27.898932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:27.898953 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:34:27.899093 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:27.899097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:27.899179 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:27.899262 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:27.900629 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.901458 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.902770 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:27.903170 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:27.903267 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:27.907847 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:27.907867 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:27.907886 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:27.907931 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:27.910632 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.911354 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.911360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.911363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.911526 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:27.911892 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:27.911895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:27.911898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:27.912272 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:27.912512 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:27.912851 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 00:34:27.913091 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:34:27.914268 17572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:28.003824 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 17296
I20260812 00:34:28.032231 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 16575
I20260812 00:34:28.033537 16693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:28.144140 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 16575
2026-08-12T00:34:28Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4643 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T00:34:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:28Z Disabled control of system clock
I20260812 00:34:28.236440 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:38895
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:38895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 00:34:28.590842 17688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:28.591180 17688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:28.591266 17688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:28.598836 17688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:28.598966 17688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:28.599004 17688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:28.599061 17688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:28.599107 17688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:28.608682 17688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:38895
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:38895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:28.610146 17688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:28.612066 17688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:28.624584 17710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:28.624609 17707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:28.624658 17712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:28.625537 17688 server_base.cc:1034] running on GCE node
I20260812 00:34:28.626470 17688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:28.628175 17688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:28.629427 17688 hybrid_clock.cc:648] HybridClock initialized: now 1786494868629365 us; error 52 us; skew 500 ppm
I20260812 00:34:28.629823 17688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:28.634589 17688 webserver.cc:460] Webserver started at http://127.7.244.126:34409/ using document root <none> and password file <none>
I20260812 00:34:28.635445 17688 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:28.635577 17688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:28.635973 17688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:28.639269 17688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "77a663f6fb784f2f9ce90f517beb62e0"
format_stamp: "Formatted at 2026-08-12 00:34:28 on dist-test-slave-gjw7"
server_key: "f743720b324436c74f25a85872f0a4eb"
server_key_iv: "6bcdcbf5c40f8e38760c9b1843f3bb52"
server_key_version: "encryptionkey@0"
I20260812 00:34:28.640156 17688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "77a663f6fb784f2f9ce90f517beb62e0"
format_stamp: "Formatted at 2026-08-12 00:34:28 on dist-test-slave-gjw7"
server_key: "f743720b324436c74f25a85872f0a4eb"
server_key_iv: "6bcdcbf5c40f8e38760c9b1843f3bb52"
server_key_version: "encryptionkey@0"
I20260812 00:34:28.646116 17688 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 00:34:28.650962 17724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:28.652566 17688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260812 00:34:28.652786 17688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "77a663f6fb784f2f9ce90f517beb62e0"
format_stamp: "Formatted at 2026-08-12 00:34:28 on dist-test-slave-gjw7"
server_key: "f743720b324436c74f25a85872f0a4eb"
server_key_iv: "6bcdcbf5c40f8e38760c9b1843f3bb52"
server_key_version: "encryptionkey@0"
I20260812 00:34:28.653038 17688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:28.659636 17688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:28.660578 17688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:28.660883 17688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:28.679538 17688 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:38895
I20260812 00:34:28.679556 17837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:38895 every 8 connection(s)
I20260812 00:34:28.681046 17688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:28.685346 17840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:28.691042 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 17688
I20260812 00:34:28.691278 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:28.692090 8145 external_mini_cluster.cc:1428] Setting key dd695821186e1ced650f827258da8ec1
I20260812 00:34:28.698177 17840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0: Bootstrap starting.
I20260812 00:34:28.701845 17840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:28.702111 17845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:28.694752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.7.244.126:38895)
0812 00:34:28.696057 (+ 1305us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:28.696070 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:28.696122 (+ 52us) server_negotiation.cc:407] Connection header received
0812 00:34:28.697997 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:28.698026 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:28.698546 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:28.698984 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:28.700242 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:28.700250 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:28.700269 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:28.700289 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:28.700359 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:28.700380 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:28.700598 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:28.700763 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 00:34:28.701047 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":431,"threads_started":1}
I20260812 00:34:28.703032 17840 log.cc:824] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:28.706297 17840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0: No bootstrap required, opened a new log
I20260812 00:34:28.714453 17840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a663f6fb784f2f9ce90f517beb62e0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38895 } }
I20260812 00:34:28.714931 17840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:28.715032 17840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a663f6fb784f2f9ce90f517beb62e0, State: Initialized, Role: FOLLOWER
I20260812 00:34:28.715802 17840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a663f6fb784f2f9ce90f517beb62e0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38895 } }
I20260812 00:34:28.716269 17840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:28.716491 17840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:28.716708 17840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:28.719231 17840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a663f6fb784f2f9ce90f517beb62e0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38895 } }
I20260812 00:34:28.719794 17840 leader_election.cc:302] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77a663f6fb784f2f9ce90f517beb62e0; no voters:
I20260812 00:34:28.720454 17840 leader_election.cc:288] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:28.720626 17848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:28.721367 17848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [term 1 LEADER]: Becoming Leader. State: Replica: 77a663f6fb784f2f9ce90f517beb62e0, State: Running, Role: LEADER
I20260812 00:34:28.722080 17848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a663f6fb784f2f9ce90f517beb62e0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38895 } }
I20260812 00:34:28.723054 17840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:28.725090 17849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77a663f6fb784f2f9ce90f517beb62e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a663f6fb784f2f9ce90f517beb62e0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38895 } } }
I20260812 00:34:28.725710 17849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:28.725116 17850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77a663f6fb784f2f9ce90f517beb62e0. Latest consensus state: current_term: 1 leader_uuid: "77a663f6fb784f2f9ce90f517beb62e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a663f6fb784f2f9ce90f517beb62e0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38895 } } }
I20260812 00:34:28.725840 17850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:28.727524 17853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:28.732568 17853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:28.747274 17853 catalog_manager.cc:1347] Generated new cluster ID: 1352f5a2e15947aeac831c8b2f8a792d
I20260812 00:34:28.747422 17853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:28.775100 17853 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:28.775702 17853 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:28.785881 17853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0: Generated new TSK 0
I20260812 00:34:28.786863 17853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:34:28.799857 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:38895
--builtin_ntp_servers=127.7.244.84:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 00:34:29.156158 17890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:29.156495 17890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:29.156608 17890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:29.164497 17890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:29.164624 17890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:29.164768 17890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:29.174358 17890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:38895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:29.175734 17890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:29.177659 17890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:29.191357 17898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:29.191396 17906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:29.191458 17901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:29.192261 17890 server_base.cc:1034] running on GCE node
I20260812 00:34:29.192952 17890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:29.194041 17890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:29.195271 17890 hybrid_clock.cc:648] HybridClock initialized: now 1786494869195217 us; error 41 us; skew 500 ppm
I20260812 00:34:29.195617 17890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:29.198041 17890 webserver.cc:460] Webserver started at http://127.7.244.65:38307/ using document root <none> and password file <none>
I20260812 00:34:29.198778 17890 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:29.198889 17890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:29.199221 17890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:29.202384 17890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "abe6cbe24d404025a9684888d5b3808c"
format_stamp: "Formatted at 2026-08-12 00:34:29 on dist-test-slave-gjw7"
server_key: "f352cabdcadc6d4813c5dc490f2e869b"
server_key_iv: "8e15e41df5e5ac4a7fb1df1153d1befb"
server_key_version: "encryptionkey@0"
I20260812 00:34:29.203373 17890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "abe6cbe24d404025a9684888d5b3808c"
format_stamp: "Formatted at 2026-08-12 00:34:29 on dist-test-slave-gjw7"
server_key: "f352cabdcadc6d4813c5dc490f2e869b"
server_key_iv: "8e15e41df5e5ac4a7fb1df1153d1befb"
server_key_version: "encryptionkey@0"
I20260812 00:34:29.209271 17890 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:34:29.213404 17916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:29.215059 17890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 00:34:29.215256 17890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "abe6cbe24d404025a9684888d5b3808c"
format_stamp: "Formatted at 2026-08-12 00:34:29 on dist-test-slave-gjw7"
server_key: "f352cabdcadc6d4813c5dc490f2e869b"
server_key_iv: "8e15e41df5e5ac4a7fb1df1153d1befb"
server_key_version: "encryptionkey@0"
I20260812 00:34:29.215523 17890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:29.222149 17890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:29.223143 17890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:29.223449 17890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:29.224649 17890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:29.226392 17890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:29.226500 17890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:29.226619 17890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:29.226688 17890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:29.259716 17890 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:32993
I20260812 00:34:29.259778 18145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:32993 every 8 connection(s)
I20260812 00:34:29.261087 17890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:29.264562 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 17890
I20260812 00:34:29.264751 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:29.265457 8145 external_mini_cluster.cc:1428] Setting key d978e097e0f6476239eff6632504acb1
I20260812 00:34:29.268442 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:38895
--builtin_ntp_servers=127.7.244.84:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 00:34:29.270057 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:29.263750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:54551 (local address 127.7.244.126:38895)
0812 00:34:29.264292 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:29.264299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:29.265702 (+ 1403us) server_negotiation.cc:407] Connection header received
0812 00:34:29.267619 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:29.267624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:29.267691 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:29.267829 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:29.269255 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:29.269262 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:29.269266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:29.269307 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:29.269365 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:29.269383 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:29.269509 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:29.269662 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:34:29.269738 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":259,"threads_started":1}
I20260812 00:34:29.270994 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:29.264371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:38895 (local address 127.7.244.65:54551)
0812 00:34:29.265510 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:29.265568 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:29.267306 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:29.268067 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:29.268077 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:29.268179 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:29.268919 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:34:29.268922 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:29.268964 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:34:29.269062 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:29.269471 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:29.269479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:29.269485 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:34:29.269715 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 00:34:29.269942 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":329,"threads_started":1}
I20260812 00:34:29.272586 18148 heartbeater.cc:347] Connected to a master server at 127.7.244.126:38895
I20260812 00:34:29.272886 18148 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:29.273574 18148 heartbeater.cc:511] Master 127.7.244.126:38895 requested a full tablet report, sending...
I20260812 00:34:29.275689 17765 ts_manager.cc:194] Registered new tserver with Master: abe6cbe24d404025a9684888d5b3808c (127.7.244.65:32993)
W20260812 00:34:29.621661 18153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:29.622020 18153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:29.622129 18153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:29.629884 18153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:29.630019 18153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:29.630134 18153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:29.639734 18153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:38895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:29.641103 18153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:29.643100 18153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:29.658317 18175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:29.658365 18170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:29.658392 18168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:29.662984 18153 server_base.cc:1034] running on GCE node
I20260812 00:34:29.663770 18153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:29.664930 18153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:29.666198 18153 hybrid_clock.cc:648] HybridClock initialized: now 1786494869666132 us; error 61 us; skew 500 ppm
I20260812 00:34:29.666597 18153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:29.669152 18153 webserver.cc:460] Webserver started at http://127.7.244.66:45769/ using document root <none> and password file <none>
I20260812 00:34:29.669939 18153 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:29.670045 18153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:29.670382 18153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:29.673660 18153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "499287e7d5fd4636ba63414e3b3d03c6"
format_stamp: "Formatted at 2026-08-12 00:34:29 on dist-test-slave-gjw7"
server_key: "16696c073d0f51450b8385ba336fd785"
server_key_iv: "310517417a5d086b0d1107e31854c092"
server_key_version: "encryptionkey@0"
I20260812 00:34:29.674559 18153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "499287e7d5fd4636ba63414e3b3d03c6"
format_stamp: "Formatted at 2026-08-12 00:34:29 on dist-test-slave-gjw7"
server_key: "16696c073d0f51450b8385ba336fd785"
server_key_iv: "310517417a5d086b0d1107e31854c092"
server_key_version: "encryptionkey@0"
I20260812 00:34:29.680728 18153 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:34:29.684907 18187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:29.686650 18153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:34:29.686853 18153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "499287e7d5fd4636ba63414e3b3d03c6"
format_stamp: "Formatted at 2026-08-12 00:34:29 on dist-test-slave-gjw7"
server_key: "16696c073d0f51450b8385ba336fd785"
server_key_iv: "310517417a5d086b0d1107e31854c092"
server_key_version: "encryptionkey@0"
I20260812 00:34:29.687135 18153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:29.693638 18153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:29.694585 18153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:29.694869 18153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:29.695943 18153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:29.697566 18153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:29.697671 18153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:29.697788 18153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:29.697860 18153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:29.730242 18153 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:44661
I20260812 00:34:29.730342 18426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:44661 every 8 connection(s)
I20260812 00:34:29.731666 18153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:29.734249 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 18153
I20260812 00:34:29.734443 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:29.735035 8145 external_mini_cluster.cc:1428] Setting key 3c43462d17257b6f21a9af901945fdaf
I20260812 00:34:29.738339 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:38895
--builtin_ntp_servers=127.7.244.84:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 00:34:29.742115 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:29.734629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:56683 (local address 127.7.244.126:38895)
0812 00:34:29.734817 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:29.734822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:29.736826 (+ 2004us) server_negotiation.cc:407] Connection header received
0812 00:34:29.739410 (+ 2584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:29.739414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:29.739481 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:29.739579 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:29.741394 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:29.741399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:29.741402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:29.741422 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:29.741469 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:29.741482 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:29.741595 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:29.741811 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 00:34:29.741871 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 00:34:29.743538 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:29.735413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:38895 (local address 127.7.244.66:56683)
0812 00:34:29.736645 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:29.736696 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:29.738989 (+ 2293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:29.739936 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:29.739950 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:29.740107 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:29.740916 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:34:29.740919 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:29.740980 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:34:29.741110 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:29.741588 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:29.741600 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:29.741608 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:34:29.741923 (+ 315us) client_negotiation.cc:240] Negotiation successful
0812 00:34:29.742242 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":381,"threads_started":1}
I20260812 00:34:29.745081 18428 heartbeater.cc:347] Connected to a master server at 127.7.244.126:38895
I20260812 00:34:29.745445 18428 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:29.746107 18428 heartbeater.cc:511] Master 127.7.244.126:38895 requested a full tablet report, sending...
I20260812 00:34:29.747882 17765 ts_manager.cc:194] Registered new tserver with Master: 499287e7d5fd4636ba63414e3b3d03c6 (127.7.244.66:44661)
W20260812 00:34:30.093134 18439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:30.093536 18439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:30.093685 18439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:30.101266 18439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:30.101447 18439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:30.101611 18439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:30.111202 18439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:38895
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:30.112625 18439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:30.114598 18439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:30.128954 18455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:30.128892 18460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:30.128849 18458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:30.130453 18439 server_base.cc:1034] running on GCE node
I20260812 00:34:30.131145 18439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:30.132185 18439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:30.133430 18439 hybrid_clock.cc:648] HybridClock initialized: now 1786494870133370 us; error 44 us; skew 500 ppm
I20260812 00:34:30.133785 18439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:30.136250 18439 webserver.cc:460] Webserver started at http://127.7.244.67:43013/ using document root <none> and password file <none>
I20260812 00:34:30.136998 18439 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:30.137112 18439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:30.137494 18439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:30.140544 18439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "98a02c7ac97744c09efab4c1c8fbdd06"
format_stamp: "Formatted at 2026-08-12 00:34:30 on dist-test-slave-gjw7"
server_key: "43bf632149e91c6cb72df3ff66929806"
server_key_iv: "0c1718913c55506504e4c78f388ea0f4"
server_key_version: "encryptionkey@0"
I20260812 00:34:30.141440 18439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "98a02c7ac97744c09efab4c1c8fbdd06"
format_stamp: "Formatted at 2026-08-12 00:34:30 on dist-test-slave-gjw7"
server_key: "43bf632149e91c6cb72df3ff66929806"
server_key_iv: "0c1718913c55506504e4c78f388ea0f4"
server_key_version: "encryptionkey@0"
I20260812 00:34:30.146986 18439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 00:34:30.150956 18468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:30.152417 18439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:34:30.152606 18439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "98a02c7ac97744c09efab4c1c8fbdd06"
format_stamp: "Formatted at 2026-08-12 00:34:30 on dist-test-slave-gjw7"
server_key: "43bf632149e91c6cb72df3ff66929806"
server_key_iv: "0c1718913c55506504e4c78f388ea0f4"
server_key_version: "encryptionkey@0"
I20260812 00:34:30.152855 18439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:30.159507 18439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:30.160491 18439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:30.160789 18439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:30.161921 18439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:30.163611 18439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:30.163720 18439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:30.163846 18439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:30.163941 18439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:30.202400 18439 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:43097
I20260812 00:34:30.202451 18681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:43097 every 8 connection(s)
I20260812 00:34:30.204226 18439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:30.205246 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 18439
I20260812 00:34:30.205470 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:30.206074 8145 external_mini_cluster.cc:1428] Setting key 6995490b63c336469d07d9d54cb8b22c
I20260812 00:34:30.214915 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.207534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:58329 (local address 127.7.244.126:38895)
0812 00:34:30.207766 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.207773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.210328 (+ 2555us) server_negotiation.cc:407] Connection header received
0812 00:34:30.212341 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.212347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.212429 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.212545 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.214098 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.214105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.214110 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.214134 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.214180 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.214197 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.214349 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.214515 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.214594 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 00:34:30.215866 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.208289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:38895 (local address 127.7.244.67:58329)
0812 00:34:30.210122 (+ 1833us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.210177 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:30.211992 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:30.212858 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:30.212869 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:30.212980 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:30.213737 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:34:30.213739 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:30.213788 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:34:30.213893 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:30.214317 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:30.214324 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:30.214330 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:34:30.214570 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 00:34:30.214806 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"thread_start_us":981,"threads_started":1}
I20260812 00:34:30.217440 18682 heartbeater.cc:347] Connected to a master server at 127.7.244.126:38895
I20260812 00:34:30.217746 18682 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:30.218377 18682 heartbeater.cc:511] Master 127.7.244.126:38895 requested a full tablet report, sending...
I20260812 00:34:30.220037 17765 ts_manager.cc:194] Registered new tserver with Master: 98a02c7ac97744c09efab4c1c8fbdd06 (127.7.244.67:43097)
I20260812 00:34:30.223184 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:34:30.237510 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.229592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60764 (local address 127.7.244.126:38895)
0812 00:34:30.235256 (+ 5664us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.235262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.235294 (+ 32us) server_negotiation.cc:407] Connection header received
0812 00:34:30.235364 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.235368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.235428 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.235552 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.236766 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.236771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.236774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.236798 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.236837 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.236850 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.236982 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.237092 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.237171 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5500}
I20260812 00:34:30.242421 17765 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:30.249742 17765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:30.279383 18698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.263424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:43097 (local address 127.0.0.1:44650)
0812 00:34:30.273332 (+ 9908us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.273376 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:30.273541 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:30.276641 (+ 3100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:30.276649 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:30.276773 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:30.277783 (+ 1010us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:34:30.277787 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:30.277824 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:34:30.277932 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:30.278617 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:30.278632 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:30.278642 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 00:34:30.278834 (+ 192us) client_negotiation.cc:240] Negotiation successful
0812 00:34:30.278941 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9593,"thread_start_us":9413,"threads_started":1}
I20260812 00:34:30.279345 18148 heartbeater.cc:503] Master 127.7.244.126:38895 was elected leader, sending a full tablet report...
I20260812 00:34:30.281913 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.264128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.7.244.67:43097)
0812 00:34:30.272824 (+ 8696us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.272844 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.273501 (+ 657us) server_negotiation.cc:407] Connection header received
0812 00:34:30.273670 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.273679 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.273939 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.274115 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.278240 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.278251 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.278269 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.278291 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.278359 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.278378 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.278555 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.279893 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.280015 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8488,"thread_start_us":8301,"threads_started":1}
I20260812 00:34:30.286729 18698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.273885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38712)
0812 00:34:30.279617 (+ 5732us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.279638 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:30.279721 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:30.284814 (+ 5093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:30.284820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:30.284862 (+ 42us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:30.285586 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:34:30.285589 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:30.285609 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:34:30.285666 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:30.286236 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:30.286241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:30.286247 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:34:30.286346 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:34:30.286410 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5626,"thread_start_us":6455,"threads_started":1}
I20260812 00:34:30.287469 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.280743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:44661 (local address 127.0.0.1:45970)
0812 00:34:30.282426 (+ 1683us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.282445 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:30.282541 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:30.285407 (+ 2866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:30.285413 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:30.285438 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:30.286114 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:34:30.286117 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:30.286133 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:34:30.286181 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:30.286681 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:30.286686 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:30.286691 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:34:30.286841 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 00:34:30.287036 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1533}
I20260812 00:34:30.288491 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.281070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45970 (local address 127.7.244.66:44661)
0812 00:34:30.283893 (+ 2823us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.283902 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.283933 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:34:30.284006 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.284015 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.284260 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.284426 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.286374 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.286382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.286395 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.286424 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.286484 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.286504 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.286628 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.288050 (+ 1422us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.288166 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2654,"thread_start_us":2522,"threads_started":1}
I20260812 00:34:30.288672 18702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.274245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.7.244.65:32993)
0812 00:34:30.283048 (+ 8803us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.283056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.283086 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:34:30.283157 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.283165 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.283415 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.283590 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.285877 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.285885 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.285897 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.285913 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.285985 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.286007 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.286183 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.287265 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.287399 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8611,"thread_start_us":9013,"threads_started":1}
I20260812 00:34:30.288514 18560 tablet_service.cc:1467] Processing CreateTablet for tablet 318d12300e5142b68cf4d9fd2c1f0256 (DEFAULT_TABLE table=test-table [id=d3e18d520a5e4ad18551f5260b2c4506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:30.290395 18560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318d12300e5142b68cf4d9fd2c1f0256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:30.292941 18295 tablet_service.cc:1467] Processing CreateTablet for tablet 318d12300e5142b68cf4d9fd2c1f0256 (DEFAULT_TABLE table=test-table [id=d3e18d520a5e4ad18551f5260b2c4506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:30.292941 18015 tablet_service.cc:1467] Processing CreateTablet for tablet 318d12300e5142b68cf4d9fd2c1f0256 (DEFAULT_TABLE table=test-table [id=d3e18d520a5e4ad18551f5260b2c4506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:30.294755 18295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318d12300e5142b68cf4d9fd2c1f0256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:30.296238 18015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318d12300e5142b68cf4d9fd2c1f0256. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:30.312527 18706 tablet_bootstrap.cc:492] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Bootstrap starting.
I20260812 00:34:30.313675 18707 tablet_bootstrap.cc:492] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Bootstrap starting.
I20260812 00:34:30.317633 18707 tablet_bootstrap.cc:654] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:30.317667 18706 tablet_bootstrap.cc:654] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:30.318495 18709 tablet_bootstrap.cc:492] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c: Bootstrap starting.
I20260812 00:34:30.319451 18706 log.cc:824] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:30.319830 18707 log.cc:824] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:30.323262 18706 tablet_bootstrap.cc:492] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: No bootstrap required, opened a new log
I20260812 00:34:30.323702 18706 ts_tablet_manager.cc:1397] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 00:34:30.323917 18707 tablet_bootstrap.cc:492] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: No bootstrap required, opened a new log
I20260812 00:34:30.324265 18707 ts_tablet_manager.cc:1397] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260812 00:34:30.324877 18709 tablet_bootstrap.cc:654] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:30.326445 18709 log.cc:824] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:30.335956 18706 raft_consensus.cc:348] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.337492 18706 raft_consensus.cc:374] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:30.337625 18706 raft_consensus.cc:729] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98a02c7ac97744c09efab4c1c8fbdd06, State: Initialized, Role: FOLLOWER
I20260812 00:34:30.337594 18707 raft_consensus.cc:348] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.338124 18707 raft_consensus.cc:374] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:30.338235 18707 raft_consensus.cc:729] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499287e7d5fd4636ba63414e3b3d03c6, State: Initialized, Role: FOLLOWER
I20260812 00:34:30.338356 18706 consensus_queue.cc:260] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.338862 18707 consensus_queue.cc:260] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.339591 18682 heartbeater.cc:503] Master 127.7.244.126:38895 was elected leader, sending a full tablet report...
I20260812 00:34:30.339797 18709 tablet_bootstrap.cc:492] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c: No bootstrap required, opened a new log
I20260812 00:34:30.340116 18709 ts_tablet_manager.cc:1397] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.012s
I20260812 00:34:30.340258 18706 ts_tablet_manager.cc:1428] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Time spent starting tablet: real 0.016s user 0.005s sys 0.009s
I20260812 00:34:30.341199 18428 heartbeater.cc:503] Master 127.7.244.126:38895 was elected leader, sending a full tablet report...
I20260812 00:34:30.343008 18707 ts_tablet_manager.cc:1428] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Time spent starting tablet: real 0.019s user 0.010s sys 0.005s
I20260812 00:34:30.350728 18709 raft_consensus.cc:348] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.351152 18709 raft_consensus.cc:374] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:30.351269 18709 raft_consensus.cc:729] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe6cbe24d404025a9684888d5b3808c, State: Initialized, Role: FOLLOWER
I20260812 00:34:30.351935 18709 consensus_queue.cc:260] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.353722 18709 ts_tablet_manager.cc:1428] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
I20260812 00:34:30.424384 18723 raft_consensus.cc:482] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:30.424721 18723 raft_consensus.cc:504] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.426677 18723 leader_election.cc:288] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98a02c7ac97744c09efab4c1c8fbdd06 (127.7.244.67:43097), abe6cbe24d404025a9684888d5b3808c (127.7.244.65:32993)
I20260812 00:34:30.430233 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.427131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:59049 (local address 127.7.244.67:43097)
0812 00:34:30.427335 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.427339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.427755 (+ 416us) server_negotiation.cc:407] Connection header received
0812 00:34:30.427994 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.428001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.428076 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.428239 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.429548 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.429555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.429561 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.429579 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.429617 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.429630 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.429772 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.429858 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.429933 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:34:30.430275 18702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.427943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:56163 (local address 127.7.244.65:32993)
0812 00:34:30.428132 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.428139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.428472 (+ 333us) server_negotiation.cc:407] Connection header received
0812 00:34:30.428610 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.428614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.428671 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.428760 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.429655 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.429661 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.429666 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.429683 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.429717 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.429729 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.429844 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.429943 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.430014 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 00:34:30.430577 18726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.426950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:43097 (local address 127.7.244.66:59049)
0812 00:34:30.427633 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.427653 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:30.427783 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:30.428350 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:30.428356 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:30.428385 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:30.429225 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:34:30.429229 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:30.429274 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:34:30.429365 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:30.429726 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:30.429731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:30.429736 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:34:30.429909 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 00:34:30.429969 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":330,"threads_started":1}
I20260812 00:34:30.430608 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.427787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.7.244.66:56163)
0812 00:34:30.428357 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.428373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:30.428481 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:30.428803 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:30.428807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:30.428830 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:30.429446 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:34:30.429449 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:30.429463 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:34:30.429503 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:30.429817 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:30.429821 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:30.429826 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:34:30.429996 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 00:34:30.430051 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":273,"threads_started":1}
I20260812 00:34:30.431885 18601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318d12300e5142b68cf4d9fd2c1f0256" candidate_uuid: "499287e7d5fd4636ba63414e3b3d03c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" is_pre_election: true
I20260812 00:34:30.431885 18048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318d12300e5142b68cf4d9fd2c1f0256" candidate_uuid: "499287e7d5fd4636ba63414e3b3d03c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe6cbe24d404025a9684888d5b3808c" is_pre_election: true
I20260812 00:34:30.432536 18048 raft_consensus.cc:2454] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 499287e7d5fd4636ba63414e3b3d03c6 in term 0.
I20260812 00:34:30.432536 18601 raft_consensus.cc:2454] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 499287e7d5fd4636ba63414e3b3d03c6 in term 0.
I20260812 00:34:30.433332 18194 leader_election.cc:302] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499287e7d5fd4636ba63414e3b3d03c6, 98a02c7ac97744c09efab4c1c8fbdd06; no voters:
I20260812 00:34:30.433795 18723 raft_consensus.cc:2781] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:30.433975 18723 raft_consensus.cc:482] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:30.434087 18723 raft_consensus.cc:3037] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:30.436422 18723 raft_consensus.cc:504] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.437117 18723 leader_election.cc:288] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [CANDIDATE]: Term 1 election: Requested vote from peers 98a02c7ac97744c09efab4c1c8fbdd06 (127.7.244.67:43097), abe6cbe24d404025a9684888d5b3808c (127.7.244.65:32993)
I20260812 00:34:30.437812 18048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318d12300e5142b68cf4d9fd2c1f0256" candidate_uuid: "499287e7d5fd4636ba63414e3b3d03c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe6cbe24d404025a9684888d5b3808c"
I20260812 00:34:30.437812 18601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318d12300e5142b68cf4d9fd2c1f0256" candidate_uuid: "499287e7d5fd4636ba63414e3b3d03c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98a02c7ac97744c09efab4c1c8fbdd06"
I20260812 00:34:30.438062 18048 raft_consensus.cc:3037] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:30.438062 18601 raft_consensus.cc:3037] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:30.441500 18601 raft_consensus.cc:2454] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 499287e7d5fd4636ba63414e3b3d03c6 in term 1.
I20260812 00:34:30.441500 18048 raft_consensus.cc:2454] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 499287e7d5fd4636ba63414e3b3d03c6 in term 1.
I20260812 00:34:30.442032 18194 leader_election.cc:302] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 499287e7d5fd4636ba63414e3b3d03c6, abe6cbe24d404025a9684888d5b3808c; no voters:
I20260812 00:34:30.442354 18723 raft_consensus.cc:2781] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:30.442840 18723 raft_consensus.cc:686] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 1 LEADER]: Becoming Leader. State: Replica: 499287e7d5fd4636ba63414e3b3d03c6, State: Running, Role: LEADER
I20260812 00:34:30.443411 18723 consensus_queue.cc:237] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } }
I20260812 00:34:30.448483 17764 catalog_manager.cc:5495] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 499287e7d5fd4636ba63414e3b3d03c6 (127.7.244.66). New cstate: current_term: 1 leader_uuid: "499287e7d5fd4636ba63414e3b3d03c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "499287e7d5fd4636ba63414e3b3d03c6" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44661 } health_report { overall_health: HEALTHY } } }
W20260812 00:34:30.456192 18683 tablet.cc:2369] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:30.467077 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.464884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45976 (local address 127.7.244.66:44661)
0812 00:34:30.465048 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.465053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.465138 (+ 85us) server_negotiation.cc:407] Connection header received
0812 00:34:30.465384 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.465389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.465480 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.465566 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.466408 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.466413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.466417 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.466434 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.466470 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.466482 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.466599 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.466740 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.466811 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:34:30.476264 18048 raft_consensus.cc:1261] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c [term 1 FOLLOWER]: Refusing update from remote peer 499287e7d5fd4636ba63414e3b3d03c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:30.476262 18601 raft_consensus.cc:1261] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 1 FOLLOWER]: Refusing update from remote peer 499287e7d5fd4636ba63414e3b3d03c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:30.477362 18736 consensus_queue.cc:1035] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe6cbe24d404025a9684888d5b3808c" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 32993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:30.477754 18723 consensus_queue.cc:1035] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98a02c7ac97744c09efab4c1c8fbdd06" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 00:34:30.483807 18430 tablet.cc:2369] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:30.504626 17764 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:30.504920 17764 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:30.509166 17764 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77a663f6fb784f2f9ce90f517beb62e0: Sending DeleteTablet for 3 replicas of tablet 318d12300e5142b68cf4d9fd2c1f0256
I20260812 00:34:30.510568 18560 tablet_service.cc:1514] Processing DeleteTablet for tablet 318d12300e5142b68cf4d9fd2c1f0256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:30 UTC) from {username='slave'} at 127.0.0.1:44650
I20260812 00:34:30.510702 18015 tablet_service.cc:1514] Processing DeleteTablet for tablet 318d12300e5142b68cf4d9fd2c1f0256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:30 UTC) from {username='slave'} at 127.0.0.1:38712
I20260812 00:34:30.510706 18295 tablet_service.cc:1514] Processing DeleteTablet for tablet 318d12300e5142b68cf4d9fd2c1f0256 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:30 UTC) from {username='slave'} at 127.0.0.1:45970
I20260812 00:34:30.513027 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 17890
I20260812 00:34:30.525029 18767 tablet_replica.cc:330] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: stopping tablet replica
I20260812 00:34:30.525848 18764 tablet_replica.cc:330] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: stopping tablet replica
I20260812 00:34:30.526702 18764 raft_consensus.cc:2229] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:30.527318 18767 raft_consensus.cc:2229] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:30.528261 18767 raft_consensus.cc:2258] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:30.529206 18764 raft_consensus.cc:2258] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:30.532819 18702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.514614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.7.244.65:32993)
0812 00:34:30.514822 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.514829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.529734 (+ 14905us) server_negotiation.cc:407] Connection header received
0812 00:34:30.530334 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.530341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.530477 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.530565 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.531945 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.531953 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.531958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.531983 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.532027 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.532045 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.532159 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.532425 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.532503 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 00:34:30.533473 18767 ts_tablet_manager.cc:1905] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:30.533870 18764 ts_tablet_manager.cc:1905] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:30.534184 18139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:30.541042 18767 ts_tablet_manager.cc:1918] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:30.541388 18767 log.cc:1196] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/318d12300e5142b68cf4d9fd2c1f0256
I20260812 00:34:30.542112 18767 ts_tablet_manager.cc:1939] T 318d12300e5142b68cf4d9fd2c1f0256 P 98a02c7ac97744c09efab4c1c8fbdd06: Deleting consensus metadata
I20260812 00:34:30.542382 18764 ts_tablet_manager.cc:1918] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:30.542683 18764 log.cc:1196] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/318d12300e5142b68cf4d9fd2c1f0256
I20260812 00:34:30.543325 18764 ts_tablet_manager.cc:1939] T 318d12300e5142b68cf4d9fd2c1f0256 P 499287e7d5fd4636ba63414e3b3d03c6: Deleting consensus metadata
I20260812 00:34:30.544575 17730 catalog_manager.cc:4841] TS 98a02c7ac97744c09efab4c1c8fbdd06 (127.7.244.67:43097): tablet 318d12300e5142b68cf4d9fd2c1f0256 (table test-table [id=d3e18d520a5e4ad18551f5260b2c4506]) successfully deleted
I20260812 00:34:30.545197 17730 catalog_manager.cc:4841] TS 499287e7d5fd4636ba63414e3b3d03c6 (127.7.244.66:44661): tablet 318d12300e5142b68cf4d9fd2c1f0256 (table test-table [id=d3e18d520a5e4ad18551f5260b2c4506]) successfully deleted
I20260812 00:34:30.658551 18768 tablet_replica.cc:330] T 318d12300e5142b68cf4d9fd2c1f0256 P abe6cbe24d404025a9684888d5b3808c: stopping tablet replica
I20260812 00:34:30.658712 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 17890
W20260812 00:34:30.684295 17730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.244.65:32993 (error 108)
I20260812 00:34:30.684547 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18153
I20260812 00:34:30.687301 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.686465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38728)
0812 00:34:30.686635 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.686814 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 00:34:30.687883 17730 catalog_manager.cc:4584] TS abe6cbe24d404025a9684888d5b3808c (127.7.244.65:32993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 318d12300e5142b68cf4d9fd2c1f0256: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
I20260812 00:34:30.688711 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.686121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.7.244.66:44661)
0812 00:34:30.686326 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.686333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.686357 (+ 24us) server_negotiation.cc:407] Connection header received
0812 00:34:30.686456 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.686461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.686526 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.686644 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.687752 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.687758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.687763 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.687783 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.687830 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.687846 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.687976 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.688404 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.688470 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:34:30.689502 18419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:30.741686 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.741116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38742)
0812 00:34:30.741388 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.741505 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 00:34:30.742957 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.742467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38748)
0812 00:34:30.742711 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.742805 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260812 00:34:30.802986 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 18153
I20260812 00:34:30.803596 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.803109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38752)
0812 00:34:30.803304 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.803400 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 00:34:30.808101 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.807494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38768)
0812 00:34:30.807739 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.807871 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 00:34:30.839082 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18439
I20260812 00:34:30.841979 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.839884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44660 (local address 127.7.244.67:43097)
0812 00:34:30.840068 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:30.840074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:30.840093 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:34:30.840218 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:30.840222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:30.840302 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:30.840384 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:30.841269 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:30.841273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:30.841276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:30.841320 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:30.841362 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:30.841375 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:30.841521 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:30.841642 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 00:34:30.841693 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:34:30.842674 18670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:30.889364 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.888692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38774)
0812 00:34:30.888961 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.889110 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 00:34:30.890610 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:30.890119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38776)
0812 00:34:30.890337 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:30.890447 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 00:34:30.931802 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 18439
I20260812 00:34:30.956578 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 17688
I20260812 00:34:30.957896 17827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:31.064877 18700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:31.064209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:32993 (local address 127.0.0.1:38786)
0812 00:34:31.064568 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:31.064669 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:32993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
I20260812 00:34:31.064942 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 17688
2026-08-12T00:34:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2920 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 00:34:31.125228 8145 mini_oidc.cc:152] Starting JWKS server
I20260812 00:34:31.128335 8145 webserver.cc:460] Webserver started at https://127.0.0.1:40175/ using document root <none> and password file <none>
I20260812 00:34:31.128705 8145 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 00:34:31.129522 8145 webserver.cc:460] Webserver started at http://127.0.0.1:41341/ using document root <none> and password file <none>
2026-08-12T00:34:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:31Z Disabled control of system clock
I20260812 00:34:31.164179 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44235
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:44235
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40175/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/testchainca.cert with env {}
W20260812 00:34:31.518232 18793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:31.518594 18793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:31.518738 18793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:31.526253 18793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:31.526405 18793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:31.526461 18793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:31.526541 18793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:31.526603 18793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:34:31.526692 18793 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 00:34:31.526742 18793 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40175/jwks.json
I20260812 00:34:31.537432 18793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:44235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40175/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:31.538883 18793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:31.540311 18793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:31.553448 18803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:31.553501 18809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:31.553447 18804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:31.554533 18793 server_base.cc:1034] running on GCE node
I20260812 00:34:31.555474 18793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:31.557205 18793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:31.558501 18793 hybrid_clock.cc:648] HybridClock initialized: now 1786494871558446 us; error 44 us; skew 500 ppm
I20260812 00:34:31.558879 18793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:31.567552 18793 webserver.cc:460] Webserver started at http://127.7.244.126:36407/ using document root <none> and password file <none>
I20260812 00:34:31.568408 18793 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:31.568521 18793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:31.568806 18793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:31.572096 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "1a3285f90b7f43b5a5d41bff96d19860"
format_stamp: "Formatted at 2026-08-12 00:34:31 on dist-test-slave-gjw7"
server_key: "75ee5e7c526da5e4ad63489e9b845851"
server_key_iv: "24a0ac18b2b380ad51acf555647a5184"
server_key_version: "encryptionkey@0"
I20260812 00:34:31.572978 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "1a3285f90b7f43b5a5d41bff96d19860"
format_stamp: "Formatted at 2026-08-12 00:34:31 on dist-test-slave-gjw7"
server_key: "75ee5e7c526da5e4ad63489e9b845851"
server_key_iv: "24a0ac18b2b380ad51acf555647a5184"
server_key_version: "encryptionkey@0"
I20260812 00:34:31.579370 18793 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 00:34:31.583668 18823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:31.585559 18793 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:34:31.585783 18793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "1a3285f90b7f43b5a5d41bff96d19860"
format_stamp: "Formatted at 2026-08-12 00:34:31 on dist-test-slave-gjw7"
server_key: "75ee5e7c526da5e4ad63489e9b845851"
server_key_iv: "24a0ac18b2b380ad51acf555647a5184"
server_key_version: "encryptionkey@0"
I20260812 00:34:31.586087 18793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:31.622449 18793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:31.623397 18793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:31.623724 18793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:31.642712 18793 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:44235
I20260812 00:34:31.642730 18927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:44235 every 8 connection(s)
I20260812 00:34:31.644182 18793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:31.648165 18931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:31.649739 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 18793
I20260812 00:34:31.649935 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:31.650578 8145 external_mini_cluster.cc:1428] Setting key 5fc4745678478fce874962b4b1ae727b
I20260812 00:34:31.662665 18931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860: Bootstrap starting.
I20260812 00:34:31.666385 18931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:31.667614 18931 log.cc:824] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:31.667687 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:31.652629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.7.244.126:44235)
0812 00:34:31.653883 (+ 1254us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:31.653901 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:31.653963 (+ 62us) server_negotiation.cc:407] Connection header received
0812 00:34:31.656443 (+ 2480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:31.656478 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:31.657166 (+ 688us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:31.657706 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:31.658700 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:31.660984 (+ 2284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:31.662524 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:31.663357 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:31.663563 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:31.664970 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:31.664979 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:31.664996 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:31.665646 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:31.665749 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:31.665775 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:31.665890 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:31.666191 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 00:34:31.666622 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":354,"threads_started":1}
I20260812 00:34:31.671099 18931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860: No bootstrap required, opened a new log
I20260812 00:34:31.672408 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:34:31.679067 18931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3285f90b7f43b5a5d41bff96d19860" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44235 } }
I20260812 00:34:31.679497 18931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:31.679592 18931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a3285f90b7f43b5a5d41bff96d19860, State: Initialized, Role: FOLLOWER
I20260812 00:34:31.680199 18931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3285f90b7f43b5a5d41bff96d19860" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44235 } }
I20260812 00:34:31.680434 18931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:31.680567 18931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:31.680734 18931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:31.683459 18931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3285f90b7f43b5a5d41bff96d19860" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44235 } }
I20260812 00:34:31.684036 18931 leader_election.cc:302] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a3285f90b7f43b5a5d41bff96d19860; no voters:
I20260812 00:34:31.684635 18931 leader_election.cc:288] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:31.684780 18943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:31.685501 18943 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [term 1 LEADER]: Becoming Leader. State: Replica: 1a3285f90b7f43b5a5d41bff96d19860, State: Running, Role: LEADER
I20260812 00:34:31.686164 18943 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3285f90b7f43b5a5d41bff96d19860" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44235 } }
I20260812 00:34:31.686841 18931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:31.689106 18946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a3285f90b7f43b5a5d41bff96d19860. Latest consensus state: current_term: 1 leader_uuid: "1a3285f90b7f43b5a5d41bff96d19860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3285f90b7f43b5a5d41bff96d19860" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44235 } } }
I20260812 00:34:31.689633 18946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:31.689534 18945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a3285f90b7f43b5a5d41bff96d19860" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a3285f90b7f43b5a5d41bff96d19860" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44235 } } }
I20260812 00:34:31.689962 18945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:31.690675 18955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:31.696285 18955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:31.706200 18955 catalog_manager.cc:1347] Generated new cluster ID: 9a51360d58e943ea8b3c14d61755ab90
I20260812 00:34:31.706341 18955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:31.727001 18955 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:31.729125 18955 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:31.745164 18955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a3285f90b7f43b5a5d41bff96d19860: Generated new TSK 0
I20260812 00:34:31.746546 18955 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 00:34:31.789935 18992 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 00:34:31.790264 18992 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 00:34:31.792608 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:31.767592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.7.244.126:44235)
0812 00:34:31.767829 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:31.767837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:31.768331 (+ 494us) server_negotiation.cc:407] Connection header received
0812 00:34:31.768779 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:31.768784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:31.768866 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:31.769017 (+ 151us) server_negotiation.cc:226] Negotiated authn=JWT
0812 00:34:31.769682 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:31.770883 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:31.772457 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:31.773214 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:31.773382 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:31.774091 (+ 709us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 00:34:31.791824 (+ 17733us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40175/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 00:34:31.792279 (+ 455us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58826: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40175/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 00:34:31.792745 18937 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58826: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40175/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 00:34:31.794396 18991 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:31.767489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44235 (local address 127.0.0.1:58826)
0812 00:34:31.768200 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:31.768219 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:31.768430 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:31.769065 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:31.769070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:31.769084 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0812 00:34:31.769483 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:31.769495 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:31.771090 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:31.771096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:31.772279 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:31.772294 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:31.773476 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:31.773483 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:31.773648 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:31.773743 (+ 95us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 00:34:31.792224 (+ 18481us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40175/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 00:34:31.792550 (+ 326us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.244.126:44235: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40175/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":361,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.244.126:44235: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40175/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 00:34:31.796653 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18793
I20260812 00:34:31.797994 18917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:31.897817 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 18793
2026-08-12T00:34:31Z chronyd exiting
I20260812 00:34:31.960460 8145 test_util.cc:175] -----------------------------------------------
I20260812 00:34:31.960620 8145 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786494824837501-8145-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (840 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 00:34:31.967597 8145 mini_oidc.cc:152] Starting JWKS server
I20260812 00:34:31.969631 8145 webserver.cc:460] Webserver started at https://127.0.0.1:34903/ using document root <none> and password file <none>
I20260812 00:34:31.969868 8145 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 00:34:31.970629 8145 webserver.cc:460] Webserver started at http://127.0.0.1:41453/ using document root <none> and password file <none>
2026-08-12T00:34:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:31Z Disabled control of system clock
I20260812 00:34:32.005028 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:37261
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:37261
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34903/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/testchainca.cert with env {}
W20260812 00:34:32.359156 19007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:32.359505 19007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:32.359640 19007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:32.367206 19007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:32.367357 19007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:32.367426 19007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:32.367513 19007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:32.367576 19007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:34:32.367664 19007 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 00:34:32.367719 19007 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34903/jwks.json
I20260812 00:34:32.378669 19007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:37261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:37261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34903/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:32.380148 19007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:32.381573 19007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:32.393791 19017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:32.393821 19025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:32.393872 19021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:32.394650 19007 server_base.cc:1034] running on GCE node
I20260812 00:34:32.395469 19007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:32.397079 19007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:32.398303 19007 hybrid_clock.cc:648] HybridClock initialized: now 1786494872398256 us; error 79 us; skew 500 ppm
I20260812 00:34:32.398680 19007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:32.401116 19007 webserver.cc:460] Webserver started at http://127.7.244.126:36695/ using document root <none> and password file <none>
I20260812 00:34:32.401896 19007 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:32.402011 19007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:32.402347 19007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:32.405435 19007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "b8a912f0845f452faf76f0332f6a7c9b"
format_stamp: "Formatted at 2026-08-12 00:34:32 on dist-test-slave-gjw7"
server_key: "4df4dd2a9b64c6984e5bc8e0c15abdfb"
server_key_iv: "9493ae1008de72fb49b855289ac73626"
server_key_version: "encryptionkey@0"
I20260812 00:34:32.406338 19007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "b8a912f0845f452faf76f0332f6a7c9b"
format_stamp: "Formatted at 2026-08-12 00:34:32 on dist-test-slave-gjw7"
server_key: "4df4dd2a9b64c6984e5bc8e0c15abdfb"
server_key_iv: "9493ae1008de72fb49b855289ac73626"
server_key_version: "encryptionkey@0"
I20260812 00:34:32.412097 19007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:34:32.416155 19039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:32.417832 19007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 00:34:32.418053 19007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "b8a912f0845f452faf76f0332f6a7c9b"
format_stamp: "Formatted at 2026-08-12 00:34:32 on dist-test-slave-gjw7"
server_key: "4df4dd2a9b64c6984e5bc8e0c15abdfb"
server_key_iv: "9493ae1008de72fb49b855289ac73626"
server_key_version: "encryptionkey@0"
I20260812 00:34:32.418367 19007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:32.432548 19007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:32.433559 19007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:32.433909 19007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:32.452916 19007 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:37261
I20260812 00:34:32.452952 19136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:37261 every 8 connection(s)
I20260812 00:34:32.454461 19007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:32.459043 19137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:32.459558 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19007
I20260812 00:34:32.459754 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:32.460412 8145 external_mini_cluster.cc:1428] Setting key 67def700b14eecb26471e2caeb7097d1
I20260812 00:34:32.475669 19137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b: Bootstrap starting.
I20260812 00:34:32.478385 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:32.462520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46236 (local address 127.7.244.126:37261)
0812 00:34:32.463774 (+ 1254us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:32.463791 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:32.463854 (+ 63us) server_negotiation.cc:407] Connection header received
0812 00:34:32.466342 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:32.466379 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:32.467128 (+ 749us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:32.467675 (+ 547us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:32.468254 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:32.470278 (+ 2024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:32.471839 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:32.473131 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:32.473410 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:32.474867 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:32.474881 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:32.474905 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:32.475753 (+ 848us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:32.475890 (+ 137us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:32.475943 (+ 53us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:32.476093 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:32.476409 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 00:34:32.476830 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":361,"threads_started":1}
I20260812 00:34:32.480321 19137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:32.481806 19137 log.cc:824] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:32.483317 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:34:32.485133 19137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b: No bootstrap required, opened a new log
I20260812 00:34:32.494766 19137 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a912f0845f452faf76f0332f6a7c9b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37261 } }
I20260812 00:34:32.495272 19137 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:32.495393 19137 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a912f0845f452faf76f0332f6a7c9b, State: Initialized, Role: FOLLOWER
I20260812 00:34:32.496129 19137 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a912f0845f452faf76f0332f6a7c9b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37261 } }
I20260812 00:34:32.496394 19137 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:32.496546 19137 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:32.496714 19137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:32.499866 19137 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a912f0845f452faf76f0332f6a7c9b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37261 } }
I20260812 00:34:32.500489 19137 leader_election.cc:302] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8a912f0845f452faf76f0332f6a7c9b; no voters:
I20260812 00:34:32.501315 19137 leader_election.cc:288] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:32.501470 19150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:32.502385 19150 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [term 1 LEADER]: Becoming Leader. State: Replica: b8a912f0845f452faf76f0332f6a7c9b, State: Running, Role: LEADER
I20260812 00:34:32.502946 19150 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a912f0845f452faf76f0332f6a7c9b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37261 } }
I20260812 00:34:32.503593 19137 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:32.505818 19153 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8a912f0845f452faf76f0332f6a7c9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a912f0845f452faf76f0332f6a7c9b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37261 } } }
I20260812 00:34:32.506268 19153 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:32.506217 19154 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8a912f0845f452faf76f0332f6a7c9b. Latest consensus state: current_term: 1 leader_uuid: "b8a912f0845f452faf76f0332f6a7c9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a912f0845f452faf76f0332f6a7c9b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37261 } } }
I20260812 00:34:32.506583 19154 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:32.510087 19160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:32.515946 19160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:32.525843 19160 catalog_manager.cc:1347] Generated new cluster ID: 7d183fbe0d2645edbf9002f69ed92ed8
I20260812 00:34:32.525977 19160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:32.537276 19160 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:32.538769 19160 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:32.548592 19160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8a912f0845f452faf76f0332f6a7c9b: Generated new TSK 0
I20260812 00:34:32.549597 19160 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 00:34:32.588757 19197 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 00:34:32.588898 19197 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 00:34:32.590173 19147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:32.569450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.7.244.126:37261)
0812 00:34:32.569658 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:32.569665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:32.570039 (+ 374us) server_negotiation.cc:407] Connection header received
0812 00:34:32.570299 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:32.570305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:32.570412 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:32.570597 (+ 185us) server_negotiation.cc:226] Negotiated authn=JWT
0812 00:34:32.571047 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:32.572196 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:32.573854 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:32.574564 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:32.574713 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:32.575361 (+ 648us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 00:34:32.589417 (+ 14056us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34903/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 00:34:32.589849 (+ 432us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46242: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34903/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 00:34:32.590332 19147 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46242: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34903/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 00:34:32.590386 19196 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:32.569229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:37261 (local address 127.0.0.1:46242)
0812 00:34:32.569919 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:32.569939 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:32.570082 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:32.570608 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:32.570612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:32.570623 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0812 00:34:32.570892 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:32.570902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:32.572369 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:32.572374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:32.573664 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:32.573679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:32.574817 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:32.574823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:32.574978 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:32.575021 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 00:34:32.589683 (+ 14662us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34903/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 00:34:32.589995 (+ 312us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.244.126:37261: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34903/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":398,"threads_started":1}
I20260812 00:34:32.592146 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19007
I20260812 00:34:32.593580 19129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:32.688387 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19007
2026-08-12T00:34:32Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (797 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 00:34:32.761983 8145 mini_oidc.cc:152] Starting JWKS server
I20260812 00:34:32.764030 8145 webserver.cc:460] Webserver started at https://127.0.0.1:41997/ using document root <none> and password file <none>
I20260812 00:34:32.764326 8145 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 00:34:32.765053 8145 webserver.cc:460] Webserver started at http://127.0.0.1:43443/ using document root <none> and password file <none>
2026-08-12T00:34:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:32Z Disabled control of system clock
I20260812 00:34:32.800617 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42283
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:33435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42283
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41997/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 00:34:33.153242 19210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:33.153625 19210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:33.153759 19210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:33.161278 19210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:33.161448 19210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:33.161511 19210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:33.161587 19210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:33.161653 19210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:34:33.161741 19210 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 00:34:33.161794 19210 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41997/jwks.json
I20260812 00:34:33.172354 19210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41997/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:33.174054 19210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:33.176129 19210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:33.188985 19218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:33.189093 19222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:33.188858 19217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:33.190464 19210 server_base.cc:1034] running on GCE node
I20260812 00:34:33.191407 19210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:33.193007 19210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:33.194304 19210 hybrid_clock.cc:648] HybridClock initialized: now 1786494873194190 us; error 54 us; skew 500 ppm
I20260812 00:34:33.194685 19210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:33.197237 19210 webserver.cc:460] Webserver started at http://127.7.244.126:46211/ using document root <none> and password file <none>
I20260812 00:34:33.198087 19210 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:33.198202 19210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:33.198567 19210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:33.202026 19210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "8c0e02ed5c5b4b768cd150a14da453b0"
format_stamp: "Formatted at 2026-08-12 00:34:33 on dist-test-slave-gjw7"
server_key: "14d2732b98bff914efea9136a50d76be"
server_key_iv: "f0a123e5a28319e10eeb739f3df92e44"
server_key_version: "encryptionkey@0"
I20260812 00:34:33.203016 19210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "8c0e02ed5c5b4b768cd150a14da453b0"
format_stamp: "Formatted at 2026-08-12 00:34:33 on dist-test-slave-gjw7"
server_key: "14d2732b98bff914efea9136a50d76be"
server_key_iv: "f0a123e5a28319e10eeb739f3df92e44"
server_key_version: "encryptionkey@0"
I20260812 00:34:33.209262 19210 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:34:33.213584 19239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:33.215303 19210 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260812 00:34:33.215523 19210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "8c0e02ed5c5b4b768cd150a14da453b0"
format_stamp: "Formatted at 2026-08-12 00:34:33 on dist-test-slave-gjw7"
server_key: "14d2732b98bff914efea9136a50d76be"
server_key_iv: "f0a123e5a28319e10eeb739f3df92e44"
server_key_version: "encryptionkey@0"
I20260812 00:34:33.215765 19210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:33.234220 19210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:33.235213 19210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:33.235538 19210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:33.253510 19210 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:42283
I20260812 00:34:33.253528 19340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:42283 every 8 connection(s)
I20260812 00:34:33.254995 19210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:33.256143 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19210
I20260812 00:34:33.256408 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:33.257113 8145 external_mini_cluster.cc:1428] Setting key 3ef85901b295d33ec5c0bb1c8f275c94
I20260812 00:34:33.259631 19341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:33.272187 19341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0: Bootstrap starting.
I20260812 00:34:33.272347 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:33.259235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.7.244.126:42283)
0812 00:34:33.260379 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:33.260394 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:33.260443 (+ 49us) server_negotiation.cc:407] Connection header received
0812 00:34:33.262194 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:33.262221 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:33.262837 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:33.263480 (+ 643us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:33.264174 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:33.265698 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:33.267216 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:33.267963 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:33.268166 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:33.269581 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:33.269591 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:33.269614 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:33.270313 (+ 699us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:33.270408 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:33.270434 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:33.270564 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:33.270875 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 00:34:33.271250 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":414,"threads_started":1}
I20260812 00:34:33.276300 19341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:33.277029 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:34:33.277626 19341 log.cc:824] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:33.281179 19341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0: No bootstrap required, opened a new log
I20260812 00:34:33.288987 19341 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0e02ed5c5b4b768cd150a14da453b0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42283 } }
I20260812 00:34:33.289480 19341 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:33.289578 19341 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c0e02ed5c5b4b768cd150a14da453b0, State: Initialized, Role: FOLLOWER
I20260812 00:34:33.290194 19341 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0e02ed5c5b4b768cd150a14da453b0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42283 } }
I20260812 00:34:33.290447 19341 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:33.290589 19341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:33.290771 19341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:33.293077 19341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0e02ed5c5b4b768cd150a14da453b0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42283 } }
I20260812 00:34:33.293692 19341 leader_election.cc:302] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c0e02ed5c5b4b768cd150a14da453b0; no voters:
I20260812 00:34:33.294322 19341 leader_election.cc:288] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:33.294523 19352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:33.295212 19352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c0e02ed5c5b4b768cd150a14da453b0, State: Running, Role: LEADER
I20260812 00:34:33.295871 19352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0e02ed5c5b4b768cd150a14da453b0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42283 } }
I20260812 00:34:33.296653 19341 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:33.299121 19355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c0e02ed5c5b4b768cd150a14da453b0. Latest consensus state: current_term: 1 leader_uuid: "8c0e02ed5c5b4b768cd150a14da453b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0e02ed5c5b4b768cd150a14da453b0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42283 } } }
I20260812 00:34:33.299057 19354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c0e02ed5c5b4b768cd150a14da453b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c0e02ed5c5b4b768cd150a14da453b0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42283 } } }
I20260812 00:34:33.299621 19354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:33.299621 19355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:33.304702 19361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:33.310528 19361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:33.320331 19361 catalog_manager.cc:1347] Generated new cluster ID: ec496fd95f8449238f2214c10a401ffa
I20260812 00:34:33.320619 19361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:33.336421 19361 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:33.337865 19361 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:33.344543 19361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c0e02ed5c5b4b768cd150a14da453b0: Generated new TSK 0
I20260812 00:34:33.345811 19361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:34:33.375422 19350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:33.365834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.7.244.126:42283)
0812 00:34:33.366126 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:33.366134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:33.366629 (+ 495us) server_negotiation.cc:407] Connection header received
0812 00:34:33.366936 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:33.366941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:33.367026 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:33.367179 (+ 153us) server_negotiation.cc:226] Negotiated authn=JWT
0812 00:34:33.367804 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:33.368748 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:33.371096 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:33.371597 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:33.371713 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:33.372369 (+ 656us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 00:34:33.374747 (+ 2378us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41997/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 00:34:33.375102 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37604: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41997/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 00:34:33.375568 19350 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37604: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41997/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 00:34:33.377756 19406 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:33.365747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42283 (local address 127.0.0.1:37604)
0812 00:34:33.366475 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:33.366496 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:33.366675 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:33.367230 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:33.367235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:33.367250 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0812 00:34:33.367589 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:33.367601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:33.368943 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:33.368950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:33.370865 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:33.370886 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:33.371817 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:33.371824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:33.372022 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:33.372075 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 00:34:33.374999 (+ 2924us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41997/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 00:34:33.377187 (+ 2188us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.244.126:42283: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41997/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":1801,"thread_start_us":365,"threads_started":1}
I20260812 00:34:33.379482 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19210
I20260812 00:34:33.380903 19333 generic_service.cc:196] Checking for leaks (request via RPC)
E20260812 00:34:33.387298 19407 webserver.cc:533] Webserver: sslize failed:
E20260812 00:34:33.387418 19407 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 00:34:33.474232 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19210
2026-08-12T00:34:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (775 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T00:34:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:33Z Disabled control of system clock
I20260812 00:34:33.571781 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:38801
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:36487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:38801
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/testchainca.cert with env {}
W20260812 00:34:33.927775 19418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:33.928135 19418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:33.928254 19418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:33.935995 19418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:33.936127 19418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:33.936172 19418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:33.936254 19418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:33.936308 19418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 00:34:33.936409 19418 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 00:34:33.936460 19418 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 00:34:33.947062 19418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:38801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:38801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:33.948559 19418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:33.950019 19418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:33.962641 19435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:33.962612 19442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:33.962668 19427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:33.963382 19418 server_base.cc:1034] running on GCE node
I20260812 00:34:33.964260 19418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:33.965932 19418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:33.967199 19418 hybrid_clock.cc:648] HybridClock initialized: now 1786494873967141 us; error 40 us; skew 500 ppm
I20260812 00:34:33.967648 19418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:34:33.970284 19418 webserver.cc:460] Webserver started at http://127.7.244.126:45951/ using document root <none> and password file <none>
I20260812 00:34:33.971081 19418 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:33.971202 19418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:33.971539 19418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:33.974803 19418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "fa1ec250b1604031b5668cd67cde2e60"
format_stamp: "Formatted at 2026-08-12 00:34:33 on dist-test-slave-gjw7"
server_key: "9798bab32bc7693a8e50bab29d57e1be"
server_key_iv: "5873f6808d4bfac8a6d1a97e0ba8b385"
server_key_version: "encryptionkey@0"
I20260812 00:34:33.975741 19418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "fa1ec250b1604031b5668cd67cde2e60"
format_stamp: "Formatted at 2026-08-12 00:34:33 on dist-test-slave-gjw7"
server_key: "9798bab32bc7693a8e50bab29d57e1be"
server_key_iv: "5873f6808d4bfac8a6d1a97e0ba8b385"
server_key_version: "encryptionkey@0"
I20260812 00:34:33.981365 19418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 00:34:33.984938 19455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:33.986460 19418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:34:33.986649 19418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "fa1ec250b1604031b5668cd67cde2e60"
format_stamp: "Formatted at 2026-08-12 00:34:33 on dist-test-slave-gjw7"
server_key: "9798bab32bc7693a8e50bab29d57e1be"
server_key_iv: "5873f6808d4bfac8a6d1a97e0ba8b385"
server_key_version: "encryptionkey@0"
I20260812 00:34:33.986892 19418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:34.001077 19418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:34.002087 19418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:34.002395 19418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:34.021685 19418 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:38801
I20260812 00:34:34.021711 19559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:38801 every 8 connection(s)
I20260812 00:34:34.023150 19418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:34.026981 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19418
I20260812 00:34:34.027210 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:34.027477 19561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:34.027909 8145 external_mini_cluster.cc:1428] Setting key bdb2909901ed4310a47a9098b77dcb94
I20260812 00:34:34.043985 19561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60: Bootstrap starting.
I20260812 00:34:34.044641 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:34.030019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.7.244.126:38801)
0812 00:34:34.031244 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:34.031262 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:34.031333 (+ 71us) server_negotiation.cc:407] Connection header received
0812 00:34:34.033069 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:34.033110 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:34.033900 (+ 790us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:34.034460 (+ 560us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:34.035111 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:34.037165 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:34.038704 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:34.039815 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:34.040051 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:34.041454 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:34.041468 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:34.041489 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:34:34.042250 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:34.042369 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:34:34.042409 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:34.042533 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:34.042816 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 00:34:34.043202 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":374,"threads_started":1}
I20260812 00:34:34.047775 19561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:34.049034 19561 log.cc:824] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:34.049108 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:34:34.052418 19561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60: No bootstrap required, opened a new log
I20260812 00:34:34.060701 19561 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1ec250b1604031b5668cd67cde2e60" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38801 } }
I20260812 00:34:34.061118 19561 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:34.061236 19561 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa1ec250b1604031b5668cd67cde2e60, State: Initialized, Role: FOLLOWER
I20260812 00:34:34.061872 19561 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1ec250b1604031b5668cd67cde2e60" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38801 } }
I20260812 00:34:34.062101 19561 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:34.062237 19561 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:34.062400 19561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:34.064692 19561 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1ec250b1604031b5668cd67cde2e60" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38801 } }
I20260812 00:34:34.065265 19561 leader_election.cc:302] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa1ec250b1604031b5668cd67cde2e60; no voters:
I20260812 00:34:34.065969 19561 leader_election.cc:288] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:34.066118 19567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:34.066917 19567 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [term 1 LEADER]: Becoming Leader. State: Replica: fa1ec250b1604031b5668cd67cde2e60, State: Running, Role: LEADER
I20260812 00:34:34.067629 19567 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1ec250b1604031b5668cd67cde2e60" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38801 } }
I20260812 00:34:34.068518 19561 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:34.070835 19569 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa1ec250b1604031b5668cd67cde2e60. Latest consensus state: current_term: 1 leader_uuid: "fa1ec250b1604031b5668cd67cde2e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1ec250b1604031b5668cd67cde2e60" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38801 } } }
I20260812 00:34:34.070870 19568 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa1ec250b1604031b5668cd67cde2e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1ec250b1604031b5668cd67cde2e60" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38801 } } }
I20260812 00:34:34.071295 19569 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:34.071292 19568 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:34.072748 19577 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:34.078157 19577 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:34.089403 19577 catalog_manager.cc:1347] Generated new cluster ID: 3d34a41c57bd4254ad91f8e009aacec0
I20260812 00:34:34.089526 19577 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:34.121686 19577 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:34.123103 19577 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:34.135135 19577 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa1ec250b1604031b5668cd67cde2e60: Generated new TSK 0
I20260812 00:34:34.136160 19577 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:34:34.243054 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:34.160637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.7.244.126:38801)
0812 00:34:34.160847 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:34.160853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:34.161187 (+ 334us) server_negotiation.cc:407] Connection header received
0812 00:34:34.161465 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:34.161471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:34.161578 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:34.161776 (+ 198us) server_negotiation.cc:226] Negotiated authn=JWT
0812 00:34:34.162252 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:34.163481 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:34.165061 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:34.165820 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:34.165976 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:34.166673 (+ 697us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 00:34:34.242247 (+ 75574us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 00:34:34.242735 (+ 488us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":81}
W20260812 00:34:34.243232 19565 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36366: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 00:34:34.243358 19606 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:34.160416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:38801 (local address 127.0.0.1:36366)
0812 00:34:34.161072 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:34.161090 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:34.161238 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:34.161780 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:34.161783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:34.161794 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0812 00:34:34.162083 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:34.162092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:34.163698 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:34.163704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:34.164880 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:34.164896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:34.166073 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:34.166079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:34.166220 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:34.166260 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 00:34:34.242638 (+ 76378us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 00:34:34.242965 (+ 327us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.244.126:38801: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":334,"threads_started":1}
I20260812 00:34:34.245033 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19418
I20260812 00:34:34.246227 19550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:34.350070 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19418
2026-08-12T00:34:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (854 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (442 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (440 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:34:35 dist-test-slave-gjw7 krb5kdc[19643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:34:35 dist-test-slave-gjw7 krb5kdc[19643](info): set up 2 sockets
Aug 12 00:34:35 dist-test-slave-gjw7 krb5kdc[19643](info): commencing operation
krb5kdc: starting...
W20260812 00:34:36.546432 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:36 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:34:36.801644 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39141
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:39141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:37.156425 19669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:37.156752 19669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:37.156843 19669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:37.164368 19669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:37.164527 19669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:37.164577 19669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:37.164624 19669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:37.164674 19669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:37.175258 19669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:39141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:37.176717 19669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:37.178174 19669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:37.191323 19687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:37.191391 19684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:37.191349 19683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:37.192433 19669 server_base.cc:1034] running on GCE node
I20260812 00:34:37.193444 19669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:37.195042 19669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:37.196286 19669 hybrid_clock.cc:648] HybridClock initialized: now 1786494877196209 us; error 60 us; skew 500 ppm
I20260812 00:34:37.196641 19669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:37 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494877, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:37.203048 19669 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:34:37.204879 19669 webserver.cc:460] Webserver started at http://127.7.244.126:39319/ using document root <none> and password file <none>
I20260812 00:34:37.205746 19669 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:37.205883 19669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:37.206234 19669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:37.210067 19669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "822fc67a1e5e406b93cc43109805082a"
format_stamp: "Formatted at 2026-08-12 00:34:37 on dist-test-slave-gjw7"
server_key: "965e6e3f39ae32c6f5a7c428c7a57140"
server_key_iv: "b0ec04cb9dd055c3162dab3440332374"
server_key_version: "encryptionkey@0"
I20260812 00:34:37.211046 19669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "822fc67a1e5e406b93cc43109805082a"
format_stamp: "Formatted at 2026-08-12 00:34:37 on dist-test-slave-gjw7"
server_key: "965e6e3f39ae32c6f5a7c428c7a57140"
server_key_iv: "b0ec04cb9dd055c3162dab3440332374"
server_key_version: "encryptionkey@0"
I20260812 00:34:37.217082 19669 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:34:37.221086 19702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:37.222806 19669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:34:37.223043 19669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "822fc67a1e5e406b93cc43109805082a"
format_stamp: "Formatted at 2026-08-12 00:34:37 on dist-test-slave-gjw7"
server_key: "965e6e3f39ae32c6f5a7c428c7a57140"
server_key_iv: "b0ec04cb9dd055c3162dab3440332374"
server_key_version: "encryptionkey@0"
I20260812 00:34:37.223279 19669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:37.240058 19669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:37.245035 19669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:37.245425 19669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:37.265527 19669 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:39141
I20260812 00:34:37.265671 19799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:39141 every 8 connection(s)
I20260812 00:34:37.267143 19669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:37.271505 19800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:37.277742 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19669
I20260812 00:34:37.277992 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:37.278791 8145 external_mini_cluster.cc:1428] Setting key bc744415138418ecdf8dee02ed8f5b6a
I20260812 00:34:37.284314 19800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a: Bootstrap starting.
I20260812 00:34:37.288343 19800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:37.289825 19800 log.cc:824] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:37.293895 19800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a: No bootstrap required, opened a new log
Aug 12 00:34:37 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:37.303462 19800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822fc67a1e5e406b93cc43109805082a" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39141 } }
I20260812 00:34:37.304013 19800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:37.304134 19800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 822fc67a1e5e406b93cc43109805082a, State: Initialized, Role: FOLLOWER
I20260812 00:34:37.304965 19800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822fc67a1e5e406b93cc43109805082a" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39141 } }
I20260812 00:34:37.305222 19800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:37.305413 19800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:37.305595 19800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:37.306030 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:37.281154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.7.244.126:39141)
0812 00:34:37.282178 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:37.282191 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:37.282235 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:34:37.284111 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:37.284139 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:37.284639 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:37.285022 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:37.287049 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:37.288513 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:37.290418 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:37.291589 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:37.291782 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:37.297007 (+ 5225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:37.297017 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:37.297043 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:37.297094 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:37.300671 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:37.301787 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:37.301796 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:37.301807 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:37.302060 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:37.302571 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:37.302577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:37.302579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:37.303332 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:37.303592 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:37.304296 (+ 704us) server_negotiation.cc:299] Negotiation successful
0812 00:34:37.304778 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":456,"threads_started":1}
I20260812 00:34:37.308094 19800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822fc67a1e5e406b93cc43109805082a" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39141 } }
I20260812 00:34:37.308630 19800 leader_election.cc:302] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 822fc67a1e5e406b93cc43109805082a; no voters:
I20260812 00:34:37.309343 19800 leader_election.cc:288] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:37.309432 19807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:37.310225 19807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [term 1 LEADER]: Becoming Leader. State: Replica: 822fc67a1e5e406b93cc43109805082a, State: Running, Role: LEADER
I20260812 00:34:37.310814 19807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822fc67a1e5e406b93cc43109805082a" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39141 } }
I20260812 00:34:37.311553 19800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:37.313751 19808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "822fc67a1e5e406b93cc43109805082a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822fc67a1e5e406b93cc43109805082a" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39141 } } }
I20260812 00:34:37.314170 19808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:37.314647 19809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 822fc67a1e5e406b93cc43109805082a. Latest consensus state: current_term: 1 leader_uuid: "822fc67a1e5e406b93cc43109805082a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822fc67a1e5e406b93cc43109805082a" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39141 } } }
I20260812 00:34:37.314971 19809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:37.315207 19825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:37.321926 19825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:37.334347 19825 catalog_manager.cc:1347] Generated new cluster ID: 684348a8c515414fa7d89c743d5bdb9d
I20260812 00:34:37.334501 19825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:37.353006 19825 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:37.354512 19825 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:37.365326 19825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 822fc67a1e5e406b93cc43109805082a: Generated new TSK 0
I20260812 00:34:37.366372 19825 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:34:37.430382 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39141
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:37.792270 19855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:37.792600 19855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:37.792701 19855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:37.800432 19855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:37.800571 19855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:37.800688 19855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:37.811357 19855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:37.812831 19855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:37.814446 19855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:37.828606 19865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:37.828670 19866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:37.828649 19869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:37.829608 19855 server_base.cc:1034] running on GCE node
I20260812 00:34:37.830302 19855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:37.831390 19855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:37.832635 19855 hybrid_clock.cc:648] HybridClock initialized: now 1786494877832563 us; error 64 us; skew 500 ppm
I20260812 00:34:37.832968 19855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:37 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494877, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:37.836431 19855 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:34:37.838060 19855 webserver.cc:460] Webserver started at http://127.7.244.65:42819/ using document root <none> and password file <none>
I20260812 00:34:37.838825 19855 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:37.838948 19855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:37.839278 19855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:37.842574 19855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "5f4086e52e70479090ef680c5a570534"
format_stamp: "Formatted at 2026-08-12 00:34:37 on dist-test-slave-gjw7"
server_key: "e294b77b60524152d7b9819e2fcda833"
server_key_iv: "a2afddcbe3715d22eafc2de94d1c0d97"
server_key_version: "encryptionkey@0"
I20260812 00:34:37.843470 19855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "5f4086e52e70479090ef680c5a570534"
format_stamp: "Formatted at 2026-08-12 00:34:37 on dist-test-slave-gjw7"
server_key: "e294b77b60524152d7b9819e2fcda833"
server_key_iv: "a2afddcbe3715d22eafc2de94d1c0d97"
server_key_version: "encryptionkey@0"
I20260812 00:34:37.849417 19855 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:34:37.853554 19884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:37.855185 19855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:34:37.855381 19855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "5f4086e52e70479090ef680c5a570534"
format_stamp: "Formatted at 2026-08-12 00:34:37 on dist-test-slave-gjw7"
server_key: "e294b77b60524152d7b9819e2fcda833"
server_key_iv: "a2afddcbe3715d22eafc2de94d1c0d97"
server_key_version: "encryptionkey@0"
I20260812 00:34:37.855634 19855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:37.868654 19855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:37.873509 19855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:37.873822 19855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:37.874930 19855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:37.876506 19855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:37.876587 19855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:37.876694 19855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:37.876755 19855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:37.914024 19855 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:33559
I20260812 00:34:37.914063 20097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:33559 every 8 connection(s)
I20260812 00:34:37.915483 19855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:37.918344 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19855
I20260812 00:34:37.918587 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:37.919451 8145 external_mini_cluster.cc:1428] Setting key c8be9d514a786b78fd93abb405e78219
Aug 12 00:34:37 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494877, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:37.941599 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:37.918464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:57769 (local address 127.7.244.126:39141)
0812 00:34:37.919027 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:37.919034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:37.920314 (+ 1280us) server_negotiation.cc:407] Connection header received
0812 00:34:37.922729 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:37.922733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:37.922821 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:37.922920 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:37.925286 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:37.926110 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:37.928562 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:37.929012 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:37.929141 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:37.933608 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:37.933618 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:37.933624 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:37.933679 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:37.937387 (+ 3708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:37.938427 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:37.938434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:37.938438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:37.938634 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:37.939226 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:37.939232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:37.939235 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:37.939669 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:37.939878 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:37.940791 (+ 913us) server_negotiation.cc:299] Negotiation successful
0812 00:34:37.941105 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":309,"threads_started":1}
I20260812 00:34:37.942497 20104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:37.919142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39141 (local address 127.7.244.65:57769)
0812 00:34:37.920109 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:37.920170 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:37.922350 (+ 2180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:37.923231 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:37.923246 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:37.924083 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:37.925002 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:37.925035 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:37.926332 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:37.926338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:37.928356 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:37.928378 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:37.929319 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:37.929328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:37.929637 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:37.930534 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:37.930568 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:37.933215 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:37.937611 (+ 4396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:37.937623 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:37.937648 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:37.938238 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:37.938812 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:37.938817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:37.938819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:37.939073 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:37.939859 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:37.939867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:37.940314 (+ 447us) client_negotiation.cc:769] Sending connection context
0812 00:34:37.940608 (+ 294us) client_negotiation.cc:240] Negotiation successful
0812 00:34:37.941096 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":357,"threads_started":1}
I20260812 00:34:37.944444 20098 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39141
I20260812 00:34:37.944795 20098 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:37.945674 20098 heartbeater.cc:511] Master 127.7.244.126:39141 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:34:37.948199 19730 ts_manager.cc:194] Registered new tserver with Master: 5f4086e52e70479090ef680c5a570534 (127.7.244.65:33559)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
I20260812 00:34:37.949896 19730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:57769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:34:37.980832 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39141
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:38.331226 20108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:38.331568 20108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:38.331717 20108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:38.339262 20108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:38.339421 20108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:38.339591 20108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:38.349967 20108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:38.351406 20108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:38.352867 20108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:38.367412 20127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:38.367443 20120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:38.367357 20122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:38.368731 20108 server_base.cc:1034] running on GCE node
I20260812 00:34:38.369480 20108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:38.370568 20108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:38.371831 20108 hybrid_clock.cc:648] HybridClock initialized: now 1786494878371780 us; error 57 us; skew 500 ppm
I20260812 00:34:38.372208 20108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:38 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:38.375690 20108 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:34:38.377359 20108 webserver.cc:460] Webserver started at http://127.7.244.66:41125/ using document root <none> and password file <none>
I20260812 00:34:38.378108 20108 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:38.378244 20108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:38.378572 20108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:38.381749 20108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "fd228b4b2873485d967c32c37e2edabc"
format_stamp: "Formatted at 2026-08-12 00:34:38 on dist-test-slave-gjw7"
server_key: "4a5c73ba4360ca8e257e9cbec1d1e446"
server_key_iv: "b6358780f6403eee1956e730df8b3915"
server_key_version: "encryptionkey@0"
I20260812 00:34:38.382685 20108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "fd228b4b2873485d967c32c37e2edabc"
format_stamp: "Formatted at 2026-08-12 00:34:38 on dist-test-slave-gjw7"
server_key: "4a5c73ba4360ca8e257e9cbec1d1e446"
server_key_iv: "b6358780f6403eee1956e730df8b3915"
server_key_version: "encryptionkey@0"
I20260812 00:34:38.388758 20108 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:34:38.392880 20153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:38.394487 20108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:34:38.394661 20108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "fd228b4b2873485d967c32c37e2edabc"
format_stamp: "Formatted at 2026-08-12 00:34:38 on dist-test-slave-gjw7"
server_key: "4a5c73ba4360ca8e257e9cbec1d1e446"
server_key_iv: "b6358780f6403eee1956e730df8b3915"
server_key_version: "encryptionkey@0"
I20260812 00:34:38.394905 20108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:38.415625 20108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:38.420428 20108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:38.420776 20108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:38.422053 20108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:38.423611 20108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:38.423697 20108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:38.423827 20108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:38.423892 20108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:38.455775 20108 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:35079
I20260812 00:34:38.455858 20370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:35079 every 8 connection(s)
I20260812 00:34:38.457173 20108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:38.466639 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20108
I20260812 00:34:38.466811 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:38.467248 8145 external_mini_cluster.cc:1428] Setting key 60765990694ae0a40f54b694ebfbce6c
Aug 12 00:34:38 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:38.480019 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:38.459944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:47029 (local address 127.7.244.126:39141)
0812 00:34:38.460479 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:38.460484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:38.461566 (+ 1082us) server_negotiation.cc:407] Connection header received
0812 00:34:38.463491 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:38.463496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:38.463581 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:38.463681 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:38.465407 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:38.466145 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:38.467896 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:38.468468 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:38.468603 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:38.472783 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:38.472790 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:38.472795 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:38.472842 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:38.476161 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:38.477215 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:38.477221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:38.477224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:38.477443 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:38.477961 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:38.477966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:38.477969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:38.478337 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:38.478477 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:38.479333 (+ 856us) server_negotiation.cc:299] Negotiation successful
0812 00:34:38.479617 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":252,"threads_started":1}
I20260812 00:34:38.480814 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:38.460536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39141 (local address 127.7.244.66:47029)
0812 00:34:38.461384 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:38.461434 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:38.463169 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:38.463975 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:38.463984 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:38.464558 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:38.465176 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:38.465195 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:38.466317 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:38.466321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:38.467736 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:38.467751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:38.468729 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:38.468735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:38.468960 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:38.469753 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:38.469792 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:38.472490 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:38.476404 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:38.476415 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:38.476435 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:38.477058 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:38.477565 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:38.477571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:38.477574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:38.477843 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:38.478457 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:38.478465 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:38.478932 (+ 467us) client_negotiation.cc:769] Sending connection context
0812 00:34:38.479187 (+ 255us) client_negotiation.cc:240] Negotiation successful
0812 00:34:38.479651 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":310,"threads_started":1}
I20260812 00:34:38.482481 20371 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39141
I20260812 00:34:38.482805 20371 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:38.483597 20371 heartbeater.cc:511] Master 127.7.244.126:39141 requested a full tablet report, sending...
I20260812 00:34:38.485399 19730 ts_manager.cc:194] Registered new tserver with Master: fd228b4b2873485d967c32c37e2edabc (127.7.244.66:35079)
I20260812 00:34:38.486286 19730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:47029
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:34:38.524441 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39141
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:38.872912 20389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:38.873237 20389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:38.873381 20389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:38.880892 20389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:38.881018 20389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:38.881134 20389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:38.891522 20389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:38.893010 20389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:38.894382 20389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:38.908823 20395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:38.908852 20397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:38.908937 20403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:38.909934 20389 server_base.cc:1034] running on GCE node
I20260812 00:34:38.910622 20389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:38.911705 20389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:38.912935 20389 hybrid_clock.cc:648] HybridClock initialized: now 1786494878912876 us; error 61 us; skew 500 ppm
I20260812 00:34:38.913370 20389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:38 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:38.917685 20389 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:34:38.920796 20389 webserver.cc:460] Webserver started at http://127.7.244.67:34829/ using document root <none> and password file <none>
I20260812 00:34:38.921605 20389 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:38.921741 20389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:38.922060 20389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:38.925174 20389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "6eb3d24ea1374fa49ed39736cdec59e1"
format_stamp: "Formatted at 2026-08-12 00:34:38 on dist-test-slave-gjw7"
server_key: "f0ee93754e0449c96d571c9176297927"
server_key_iv: "b41e587e2c708d4002f55e31f0ab73a5"
server_key_version: "encryptionkey@0"
I20260812 00:34:38.926111 20389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "6eb3d24ea1374fa49ed39736cdec59e1"
format_stamp: "Formatted at 2026-08-12 00:34:38 on dist-test-slave-gjw7"
server_key: "f0ee93754e0449c96d571c9176297927"
server_key_iv: "b41e587e2c708d4002f55e31f0ab73a5"
server_key_version: "encryptionkey@0"
I20260812 00:34:38.931936 20389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:34:38.936189 20420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:38.937827 20389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 00:34:38.938035 20389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "6eb3d24ea1374fa49ed39736cdec59e1"
format_stamp: "Formatted at 2026-08-12 00:34:38 on dist-test-slave-gjw7"
server_key: "f0ee93754e0449c96d571c9176297927"
server_key_iv: "b41e587e2c708d4002f55e31f0ab73a5"
server_key_version: "encryptionkey@0"
I20260812 00:34:38.938261 20389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:38.953610 20098 heartbeater.cc:503] Master 127.7.244.126:39141 was elected leader, sending a full tablet report...
I20260812 00:34:38.967746 20389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:38.972316 20389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:38.972622 20389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:38.973708 20389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:38.975172 20389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:38.975253 20389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:38.975384 20389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:38.975440 20389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:39.007223 20389 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:46495
I20260812 00:34:39.007287 20644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:46495 every 8 connection(s)
I20260812 00:34:39.008652 20389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:39.009804 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20389
I20260812 00:34:39.009987 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:39.010433 8145 external_mini_cluster.cc:1428] Setting key dac4b95f642e63e3477d36bb5c03530d
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494878, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:34:39.031428 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:39.011895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:57037 (local address 127.7.244.126:39141)
0812 00:34:39.012477 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.012483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.013878 (+ 1395us) server_negotiation.cc:407] Connection header received
0812 00:34:39.015910 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.015915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.016018 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.016112 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.018050 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.018873 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.020558 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.020972 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.021074 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:39.024813 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:39.024821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:39.024825 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:39.024873 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:39.028255 (+ 3382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.029120 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.029124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.029127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.029265 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.029689 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.029692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.029695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.029982 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:39.030107 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:39.030930 (+ 823us) server_negotiation.cc:299] Negotiation successful
0812 00:34:39.031120 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":230,"threads_started":1}
I20260812 00:34:39.032473 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:39.012612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39141 (local address 127.7.244.67:57037)
0812 00:34:39.013682 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:39.013741 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:39.015607 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:39.016357 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:39.016366 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:39.017054 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:39.017793 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:39.017816 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.019033 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.019038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:39.020366 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:39.020381 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.021172 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.021177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:39.021480 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:39.022212 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:39.022236 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:39.024504 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:39.028438 (+ 3934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:39.028447 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:39.028464 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:39.029005 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:39.029376 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:39.029379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:39.029382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:39.029595 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:39.030136 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:39.030144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:39.030537 (+ 393us) client_negotiation.cc:769] Sending connection context
0812 00:34:39.030803 (+ 266us) client_negotiation.cc:240] Negotiation successful
0812 00:34:39.031253 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":390,"threads_started":1}
I20260812 00:34:39.033972 20645 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39141
I20260812 00:34:39.034291 20645 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:39.035100 20645 heartbeater.cc:511] Master 127.7.244.126:39141 requested a full tablet report, sending...
I20260812 00:34:39.036793 19730 ts_manager.cc:194] Registered new tserver with Master: 6eb3d24ea1374fa49ed39736cdec59e1 (127.7.244.67:46495)
I20260812 00:34:39.037678 19730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:57037
I20260812 00:34:39.040184 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/test-admin.keytab.
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:39.072309 8145 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 00:34:39.078042 20668 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:39.075379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58408 (local address 127.7.244.65:33559)
0812 00:34:39.076073 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.076080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.076107 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:34:39.076171 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.076178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.076395 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.076534 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.077609 (+ 1075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58408: BlockingRecv error: recv got EOF from 127.0.0.1:58408 (error 108)
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":362,"threads_started":1}
W20260812 00:34:39.078068 8145 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5cc' of type FILE
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:39.079672 8145 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:39.081107 8145 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:34:39.098812 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:39.083015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58422 (local address 127.7.244.65:33559)
0812 00:34:39.083238 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.083245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.083515 (+ 270us) server_negotiation.cc:407] Connection header received
0812 00:34:39.083661 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.083673 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.083747 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.083845 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.084936 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.086065 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.087361 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.088001 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.088118 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:39.092141 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:39.092164 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:39.092182 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:39.092241 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:39.095379 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.096301 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.096312 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.096314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.096489 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.096904 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.096908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.096911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.097447 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:39.097702 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:39.098152 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 00:34:39.098481 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 00:34:39.105357 20668 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:39.103364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58432 (local address 127.7.244.65:33559)
0812 00:34:39.103529 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.103533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.103891 (+ 358us) server_negotiation.cc:407] Connection header received
0812 00:34:39.104018 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.104022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.104102 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.104188 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.105034 (+ 846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58432: BlockingRecv error: recv got EOF from 127.0.0.1:58432 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 00:34:39.105659 8145 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5cc' of type FILE
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:39.107261 8145 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:39.108690 8145 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:34:39.124537 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:39.110332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.7.244.65:33559)
0812 00:34:39.110535 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.110541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.110637 (+ 96us) server_negotiation.cc:407] Connection header received
0812 00:34:39.110837 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.110843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.110908 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.110990 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.112048 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.112906 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.114195 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.114650 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.114769 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:39.118777 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:39.118783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:39.118786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:39.118846 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:39.121748 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.122543 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.122549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.122551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.122703 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.123125 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.123129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.123131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.123442 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:39.123709 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:39.124023 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 00:34:39.124249 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 00:34:39.129650 20668 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:39.127703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.7.244.65:33559)
0812 00:34:39.127916 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.127921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.128142 (+ 221us) server_negotiation.cc:407] Connection header received
0812 00:34:39.128332 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.128336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.128424 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.128529 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.129425 (+ 896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58450: BlockingRecv error: recv got EOF from 127.0.0.1:58450 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 00:34:39.129819 8145 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786494824837501-8145-0/krb5kdc/krb5cc' of type FILE
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:39.131440 8145 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:39.132834 8145 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:34:39.148417 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:39.134434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58466 (local address 127.7.244.65:33559)
0812 00:34:39.134649 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.134656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.134886 (+ 230us) server_negotiation.cc:407] Connection header received
0812 00:34:39.135072 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.135076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.135140 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.135228 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.136290 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.137039 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.138316 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.138746 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.138861 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:39.142755 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:39.142761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:39.142764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:39.142807 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:39.145561 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.146311 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.146316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.146318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.146518 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.146945 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.146949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.146952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.147277 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:39.147516 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:39.147840 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 00:34:39.148039 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 00:34:39.150247 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19855
I20260812 00:34:39.163455 20668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:39.151043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58482 (local address 127.7.244.65:33559)
0812 00:34:39.151216 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.151223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.151570 (+ 347us) server_negotiation.cc:407] Connection header received
0812 00:34:39.151751 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.151755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.151819 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.151899 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.152924 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.153686 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.154942 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.155406 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.155523 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:39.157793 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:39.157799 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:39.157802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:39.157841 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:39.160536 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.161439 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.161446 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.161448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.161610 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.162001 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.162005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.162007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.162295 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:39.162561 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:39.162877 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 00:34:39.163121 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:34:39.164110 20091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:39.249097 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19855
I20260812 00:34:39.271871 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20108
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:34:39.289167 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:39.272792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.7.244.66:35079)
0812 00:34:39.273279 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.273319 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.273353 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:34:39.273446 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.273453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.273682 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.273837 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.274875 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.275934 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.277251 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.277993 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.278123 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:39.282566 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:39.282584 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:39.282602 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:39.282648 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:39.286028 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.286858 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.286866 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.286868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.287020 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.287469 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.287473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.287475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.287878 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:39.288207 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:39.288568 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 00:34:39.288850 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":228,"threads_started":1}
I20260812 00:34:39.290520 20360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:39.374373 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20108
I20260812 00:34:39.402117 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20389
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[19643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:34:39.418924 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:39.403478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.7.244.67:46495)
0812 00:34:39.404193 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:39.404200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:39.404231 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:34:39.404297 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:39.404304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:39.404531 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:39.404665 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:39.405803 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.406833 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.408085 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:39.408767 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:39.408868 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:39.413005 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:39.413020 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:39.413038 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:39.413079 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:39.416083 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.416784 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.416790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.416793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.416965 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:39.417355 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:39.417359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:39.417362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:39.417738 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:39.418034 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:39.418319 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 00:34:39.418576 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":367,"threads_started":1}
I20260812 00:34:39.420297 20636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:39.503324 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20389
I20260812 00:34:39.526458 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19669
I20260812 00:34:39.527720 19791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:39.631517 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19669
2026-08-12T00:34:39Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4410 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[20715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[20715](info): set up 2 sockets
Aug 12 00:34:39 dist-test-slave-gjw7 krb5kdc[20715](info): commencing operation
krb5kdc: starting...
W20260812 00:34:40.952884 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.009s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:41 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.126.keytab.
Entry for principal oryx/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.126.keytab.
I20260812 00:34:41.213282 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33751
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.126.keytab
--principal=oryx/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:33751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:41.573774 20737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:41.574126 20737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:41.574221 20737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:41.581789 20737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:41.581928 20737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:41.581974 20737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:41.582036 20737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:41.582083 20737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:41.592466 20737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:33751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.126.keytab
--principal=oryx/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:41.593977 20737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:41.595394 20737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:41.607828 20744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:41.607750 20751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:41.608089 20745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:41.608654 20737 server_base.cc:1034] running on GCE node
I20260812 00:34:41.609579 20737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:41.611148 20737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:41.612371 20737 hybrid_clock.cc:648] HybridClock initialized: now 1786494881612311 us; error 52 us; skew 500 ppm
I20260812 00:34:41.612735 20737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:41 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:41.616130 20737 init.cc:377] Logged in from keytab as oryx/127.7.244.126@KRBTEST.COM (short username oryx)
I20260812 00:34:41.617870 20737 webserver.cc:460] Webserver started at http://127.7.244.126:42799/ using document root <none> and password file <none>
I20260812 00:34:41.618621 20737 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:41.618757 20737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:41.619076 20737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:41.622466 20737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "57280ef1740948289e04fb0a6dd73669"
format_stamp: "Formatted at 2026-08-12 00:34:41 on dist-test-slave-gjw7"
server_key: "dcaaaf9441eadc7b49a8847c050d31f2"
server_key_iv: "45a6626d9cf5b519fdaab670599f3830"
server_key_version: "encryptionkey@0"
I20260812 00:34:41.623342 20737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "57280ef1740948289e04fb0a6dd73669"
format_stamp: "Formatted at 2026-08-12 00:34:41 on dist-test-slave-gjw7"
server_key: "dcaaaf9441eadc7b49a8847c050d31f2"
server_key_iv: "45a6626d9cf5b519fdaab670599f3830"
server_key_version: "encryptionkey@0"
I20260812 00:34:41.629402 20737 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260812 00:34:41.633486 20761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:41.635075 20737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 00:34:41.635291 20737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "57280ef1740948289e04fb0a6dd73669"
format_stamp: "Formatted at 2026-08-12 00:34:41 on dist-test-slave-gjw7"
server_key: "dcaaaf9441eadc7b49a8847c050d31f2"
server_key_iv: "45a6626d9cf5b519fdaab670599f3830"
server_key_version: "encryptionkey@0"
I20260812 00:34:41.635515 20737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:41.679934 20737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:41.684504 20737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:41.684806 20737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:41.704324 20737 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:33751
I20260812 00:34:41.704349 20858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:33751 every 8 connection(s)
I20260812 00:34:41.705950 20737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:41.710259 20859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:41.715248 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20737
I20260812 00:34:41.715481 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:41.716392 8145 external_mini_cluster.cc:1428] Setting key f68085be6bc0f6516382ae562f271bd8
I20260812 00:34:41.724063 20859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: Bootstrap starting.
I20260812 00:34:41.728322 20859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:41.729748 20859 log.cc:824] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:34:41 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:41.733804 20859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: No bootstrap required, opened a new log
I20260812 00:34:41.742290 20859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } }
I20260812 00:34:41.742584 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:41.718871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.7.244.126:33751)
0812 00:34:41.719879 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:41.719899 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:41.719962 (+ 63us) server_negotiation.cc:407] Connection header received
0812 00:34:41.722191 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:41.722233 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:41.722727 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:41.723106 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:41.725132 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:41.726658 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:41.728660 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:41.729533 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:41.729701 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:41.734614 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:41.734623 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:41.734641 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:41.734686 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:41.737884 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:41.738728 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:41.738736 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:41.738744 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:41.738962 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:41.739392 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:41.739397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:41.739399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:41.740074 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:41.740502 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:41.740894 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 00:34:41.741457 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":453,"threads_started":1}
I20260812 00:34:41.742729 20859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:41.742846 20859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57280ef1740948289e04fb0a6dd73669, State: Initialized, Role: FOLLOWER
I20260812 00:34:41.743563 20859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } }
I20260812 00:34:41.743842 20859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:41.743995 20859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:41.744184 20859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:41.746927 20859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } }
I20260812 00:34:41.747510 20859 leader_election.cc:302] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57280ef1740948289e04fb0a6dd73669; no voters:
I20260812 00:34:41.748172 20859 leader_election.cc:288] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:41.748354 20866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:41.749058 20866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 1 LEADER]: Becoming Leader. State: Replica: 57280ef1740948289e04fb0a6dd73669, State: Running, Role: LEADER
I20260812 00:34:41.749667 20866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } }
I20260812 00:34:41.750460 20859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:41.753005 20867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } }
I20260812 00:34:41.753527 20867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:41.753582 20870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57280ef1740948289e04fb0a6dd73669. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } }
I20260812 00:34:41.753930 20870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:41.756078 20875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:41.762071 20875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:41.772912 20875 catalog_manager.cc:1347] Generated new cluster ID: a6ce5fba19344004b179acb4e647ef35
I20260812 00:34:41.773042 20875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:41.803099 20875 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:41.804584 20875 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:41.817082 20875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: Generated new TSK 0
I20260812 00:34:41.818207 20875 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab.
Entry for principal oryx/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab.
I20260812 00:34:41.886281 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33751
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab
--principal=oryx/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:42.278245 20910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:42.278579 20910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:42.278702 20910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:42.286334 20910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:42.286465 20910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:42.286623 20910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:42.297005 20910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab
--principal=oryx/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:42.298483 20910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:42.299901 20910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:42.313947 20920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:42.313912 20923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:42.313874 20919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:42.314944 20910 server_base.cc:1034] running on GCE node
I20260812 00:34:42.315747 20910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:42.316895 20910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:42.318132 20910 hybrid_clock.cc:648] HybridClock initialized: now 1786494882318013 us; error 116 us; skew 500 ppm
I20260812 00:34:42.318452 20910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:42 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494882, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:42.325920 20910 init.cc:377] Logged in from keytab as oryx/127.7.244.65@KRBTEST.COM (short username oryx)
I20260812 00:34:42.327616 20910 webserver.cc:460] Webserver started at http://127.7.244.65:43321/ using document root <none> and password file <none>
I20260812 00:34:42.328402 20910 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:42.328523 20910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:42.328820 20910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:42.332100 20910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "e640fd14b3b1411fa515c1cdbd2d113d"
format_stamp: "Formatted at 2026-08-12 00:34:42 on dist-test-slave-gjw7"
server_key: "90f63719f1ea692617b0c5eee3cf1af0"
server_key_iv: "af971473152d0ab5e65dc79d944149fd"
server_key_version: "encryptionkey@0"
I20260812 00:34:42.332896 20910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "e640fd14b3b1411fa515c1cdbd2d113d"
format_stamp: "Formatted at 2026-08-12 00:34:42 on dist-test-slave-gjw7"
server_key: "90f63719f1ea692617b0c5eee3cf1af0"
server_key_iv: "af971473152d0ab5e65dc79d944149fd"
server_key_version: "encryptionkey@0"
I20260812 00:34:42.338656 20910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 00:34:42.342674 20945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:42.344388 20910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:34:42.344605 20910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "e640fd14b3b1411fa515c1cdbd2d113d"
format_stamp: "Formatted at 2026-08-12 00:34:42 on dist-test-slave-gjw7"
server_key: "90f63719f1ea692617b0c5eee3cf1af0"
server_key_iv: "af971473152d0ab5e65dc79d944149fd"
server_key_version: "encryptionkey@0"
I20260812 00:34:42.344842 20910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:42.368139 20910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:42.372666 20910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:42.372985 20910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:42.374089 20910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:42.375604 20910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:42.375689 20910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:42.375808 20910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:42.375864 20910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:42.408360 20910 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:44493
I20260812 00:34:42.408406 21157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:44493 every 8 connection(s)
I20260812 00:34:42.409857 20910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:42.411520 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20910
I20260812 00:34:42.411725 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:42.412603 8145 external_mini_cluster.cc:1428] Setting key badc1d33dbc0430c3d9aefc4c9e530da
Aug 12 00:34:42 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494882, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:42.434221 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:42.412893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:42275 (local address 127.7.244.126:33751)
0812 00:34:42.413673 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:42.413680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:42.414740 (+ 1060us) server_negotiation.cc:407] Connection header received
0812 00:34:42.416804 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:42.416810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:42.416904 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:42.417017 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:42.418719 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:42.419733 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:42.421581 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:42.422158 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:42.422300 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:42.427157 (+ 4857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:42.427164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:42.427168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:42.427212 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:42.430154 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:42.431490 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:42.431495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:42.431498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:42.431653 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:42.432213 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:42.432217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:42.432219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:42.432525 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:42.432647 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:42.433733 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0812 00:34:42.433947 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":516,"threads_started":1}
I20260812 00:34:42.435782 21165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:42.413665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:42275)
0812 00:34:42.414571 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:42.414629 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:42.416497 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:42.417226 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:42.417235 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:42.417863 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:42.418503 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:42.418531 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:42.419938 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:42.419943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:42.421385 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:42.421403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:42.422424 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:42.422430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:42.422665 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:42.423467 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:42.423506 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:42.426764 (+ 3258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:42.430396 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:42.430408 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:42.430430 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:42.431247 (+ 817us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:42.431804 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:42.431809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:42.431812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:42.432108 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:42.432654 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:42.432663 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:42.433215 (+ 552us) client_negotiation.cc:769] Sending connection context
0812 00:34:42.433623 (+ 408us) client_negotiation.cc:240] Negotiation successful
0812 00:34:42.434230 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":342,"threads_started":1}
I20260812 00:34:42.437563 21159 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33751
I20260812 00:34:42.437903 21159 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:42.438733 21159 heartbeater.cc:511] Master 127.7.244.126:33751 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:34:42.441423 20799 ts_manager.cc:194] Registered new tserver with Master: e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65:44493)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.66@KRBTEST.COM" created.
I20260812 00:34:42.443192 20799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:42275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab.
Entry for principal oryx/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab.
I20260812 00:34:42.472416 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33751
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab
--principal=oryx/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:42.835626 21169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:42.836074 21169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:42.836227 21169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:42.843923 21169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:42.844077 21169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:42.844247 21169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:42.854548 21169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab
--principal=oryx/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:42.855970 21169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:42.857403 21169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:42.871393 21176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:42.871493 21181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:42.871424 21177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:42.872066 21169 server_base.cc:1034] running on GCE node
I20260812 00:34:42.872773 21169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:42.873951 21169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:42.875190 21169 hybrid_clock.cc:648] HybridClock initialized: now 1786494882875136 us; error 47 us; skew 500 ppm
I20260812 00:34:42.875562 21169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:42 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494882, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:42.879243 21169 init.cc:377] Logged in from keytab as oryx/127.7.244.66@KRBTEST.COM (short username oryx)
I20260812 00:34:42.881078 21169 webserver.cc:460] Webserver started at http://127.7.244.66:45867/ using document root <none> and password file <none>
I20260812 00:34:42.882014 21169 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:42.882175 21169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:42.882524 21169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:42.886062 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "94e9e1797474436ab2a79319877c53dd"
format_stamp: "Formatted at 2026-08-12 00:34:42 on dist-test-slave-gjw7"
server_key: "06e97f84fbcc3ef7f2bde60ee57ea3ed"
server_key_iv: "3f219b00e75c1436abfd5355dd713e35"
server_key_version: "encryptionkey@0"
I20260812 00:34:42.887066 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "94e9e1797474436ab2a79319877c53dd"
format_stamp: "Formatted at 2026-08-12 00:34:42 on dist-test-slave-gjw7"
server_key: "06e97f84fbcc3ef7f2bde60ee57ea3ed"
server_key_iv: "3f219b00e75c1436abfd5355dd713e35"
server_key_version: "encryptionkey@0"
I20260812 00:34:42.893604 21169 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:34:42.897867 21194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:42.899493 21169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:34:42.899688 21169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "94e9e1797474436ab2a79319877c53dd"
format_stamp: "Formatted at 2026-08-12 00:34:42 on dist-test-slave-gjw7"
server_key: "06e97f84fbcc3ef7f2bde60ee57ea3ed"
server_key_iv: "3f219b00e75c1436abfd5355dd713e35"
server_key_version: "encryptionkey@0"
I20260812 00:34:42.899926 21169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:42.924723 21169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:42.929574 21169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:42.929886 21169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:42.930943 21169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:42.932448 21169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:42.932533 21169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:42.932667 21169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:42.932726 21169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:42.966679 21169 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:45217
I20260812 00:34:42.966715 21415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:45217 every 8 connection(s)
I20260812 00:34:42.968279 21169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:42.968678 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21169
I20260812 00:34:42.968860 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:42.969717 8145 external_mini_cluster.cc:1428] Setting key 2cc355aed1e614ddd897cc24cf5489c7
Aug 12 00:34:42 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494882, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:42.991578 21419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:42.971537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:48601 (local address 127.7.244.126:33751)
0812 00:34:42.972024 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:42.972030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:42.973489 (+ 1459us) server_negotiation.cc:407] Connection header received
0812 00:34:42.975514 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:42.975520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:42.975613 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:42.975714 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:42.977567 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:42.978310 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:42.979994 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:42.980450 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:42.980572 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:42.984659 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:42.984666 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:42.984669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:42.984710 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:42.987422 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:42.988569 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:42.988574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:42.988576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:42.988732 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:42.989455 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:42.989460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:42.989463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:42.989797 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:42.989939 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:42.991060 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0812 00:34:42.991283 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":230,"threads_started":1}
I20260812 00:34:42.993165 21420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:42.972316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:48601)
0812 00:34:42.973265 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:42.973351 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:42.975174 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:42.976024 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:42.976034 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:42.976622 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:42.977347 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:42.977380 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:42.978495 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:42.978499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:42.979818 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:42.979833 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:42.980713 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:42.980722 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:42.980973 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:42.981689 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:42.981715 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:42.984322 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:42.987650 (+ 3328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:42.987666 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:42.987692 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:42.988419 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:42.988859 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:42.988864 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:42.988867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:42.989185 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:42.989963 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:42.989973 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:42.990509 (+ 536us) client_negotiation.cc:769] Sending connection context
0812 00:34:42.990895 (+ 386us) client_negotiation.cc:240] Negotiation successful
0812 00:34:42.991562 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":381,"threads_started":1}
I20260812 00:34:42.994908 21417 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33751
I20260812 00:34:42.995236 21417 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.7.244.67@KRBTEST.COM; defaulting to no policy
I20260812 00:34:42.996065 21417 heartbeater.cc:511] Master 127.7.244.126:33751 requested a full tablet report, sending...
I20260812 00:34:42.997804 20799 ts_manager.cc:194] Registered new tserver with Master: 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.67@KRBTEST.COM" created.
I20260812 00:34:42.998677 20799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:48601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab.
Entry for principal oryx/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab.
I20260812 00:34:43.029955 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33751
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab
--principal=oryx/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:43.381009 21429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:43.381516 21429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:43.381736 21429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:43.389546 21429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:43.389705 21429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:43.389868 21429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:43.400352 21429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab
--principal=oryx/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33751
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:43.401839 21429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:43.403252 21429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:43.417568 21434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:43.417699 21435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:43.418385 21429 server_base.cc:1034] running on GCE node
W20260812 00:34:43.417649 21438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:43.419202 21429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:43.420258 21429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:43.421479 21429 hybrid_clock.cc:648] HybridClock initialized: now 1786494883421419 us; error 50 us; skew 500 ppm
I20260812 00:34:43.421851 21429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:43 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494883, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:43.425487 21429 init.cc:377] Logged in from keytab as oryx/127.7.244.67@KRBTEST.COM (short username oryx)
I20260812 00:34:43.427212 21429 webserver.cc:460] Webserver started at http://127.7.244.67:42997/ using document root <none> and password file <none>
I20260812 00:34:43.428043 21429 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:43.428176 21429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:43.428517 21429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:43.431914 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "1e591840d9594679926354c22614d745"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "d232a369e17d345960df2e01e4500208"
server_key_iv: "2032f7b15d15d5f6280ae30987aaea5c"
server_key_version: "encryptionkey@0"
I20260812 00:34:43.432788 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "1e591840d9594679926354c22614d745"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "d232a369e17d345960df2e01e4500208"
server_key_iv: "2032f7b15d15d5f6280ae30987aaea5c"
server_key_version: "encryptionkey@0"
I20260812 00:34:43.439049 21429 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260812 00:34:43.443493 21450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:43.445353 21429 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 00:34:43.445567 21429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "1e591840d9594679926354c22614d745"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "d232a369e17d345960df2e01e4500208"
server_key_iv: "2032f7b15d15d5f6280ae30987aaea5c"
server_key_version: "encryptionkey@0"
I20260812 00:34:43.445813 21429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:43.446694 21159 heartbeater.cc:503] Master 127.7.244.126:33751 was elected leader, sending a full tablet report...
I20260812 00:34:43.461630 21429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:43.467036 21429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:43.467372 21429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:43.468488 21429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:43.470094 21429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:43.470185 21429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:43.470309 21429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:43.470366 21429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:43.502321 21664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:44261 every 8 connection(s)
I20260812 00:34:43.502331 21429 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:44261
I20260812 00:34:43.503934 21429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:43.505995 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21429
I20260812 00:34:43.506173 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:43.506748 8145 external_mini_cluster.cc:1428] Setting key f8188943cb571e734af5042bce7a2822
Aug 12 00:34:43 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494883, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:43.526389 21667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:43.507014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:54055 (local address 127.7.244.126:33751)
0812 00:34:43.507544 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:43.507551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:43.508849 (+ 1298us) server_negotiation.cc:407] Connection header received
0812 00:34:43.510941 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:43.510946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:43.511030 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:43.511121 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:43.512843 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:43.513746 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:43.515460 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:43.515876 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:43.516012 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:43.520071 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:43.520077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:43.520081 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:43.520120 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:43.523189 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:43.524031 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:43.524036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:43.524038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:43.524181 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:43.524643 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:43.524647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:43.524650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:43.524963 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:43.525089 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:43.525889 (+ 800us) server_negotiation.cc:299] Negotiation successful
0812 00:34:43.526102 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":254,"threads_started":1}
I20260812 00:34:43.527503 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:43.507708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:54055)
0812 00:34:43.508658 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:43.508728 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:43.510657 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:43.511344 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:43.511385 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:43.511953 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:43.512610 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:43.512632 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:43.513919 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:43.513923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:43.515274 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:43.515290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:43.516133 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:43.516146 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:43.516403 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:43.517106 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:43.517131 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:43.519793 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:43.523350 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:43.523359 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:43.523377 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:43.523884 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:43.524304 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:43.524307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:43.524309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:43.524530 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:43.525065 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:43.525071 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:43.525528 (+ 457us) client_negotiation.cc:769] Sending connection context
0812 00:34:43.525784 (+ 256us) client_negotiation.cc:240] Negotiation successful
0812 00:34:43.526276 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":339,"threads_started":1}
I20260812 00:34:43.529093 21665 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33751
I20260812 00:34:43.529501 21665 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:43.530361 21665 heartbeater.cc:511] Master 127.7.244.126:33751 requested a full tablet report, sending...
I20260812 00:34:43.531977 20799 ts_manager.cc:194] Registered new tserver with Master: 1e591840d9594679926354c22614d745 (127.7.244.67:44261)
I20260812 00:34:43.532819 20799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:54055
I20260812 00:34:43.536008 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.244.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.125.keytab.
Entry for principal oryx/127.7.244.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.125.keytab.
I20260812 00:34:43.593709 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.125:40257
--webserver_interface=127.7.244.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.125.keytab
--principal=oryx/127.7.244.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:33751,127.7.244.125:40257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:43.949368 21673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:43.949733 21673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:43.949874 21673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:43.957737 21673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:43.957857 21673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:43.957911 21673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:43.957967 21673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:43.958010 21673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:43.968279 21673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:33751,127.7.244.125:40257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.125.keytab
--principal=oryx/127.7.244.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.125:40257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.244.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
W20260812 00:34:43.968688 21673 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.244.126:33751,127.7.244.125:40257'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 00:34:43.969800 21673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:43.971125 21673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:43.983666 21682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:43.983697 21681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:43.983722 21684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:43.984596 21673 server_base.cc:1034] running on GCE node
I20260812 00:34:43.985532 21673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:43.987090 21673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:43.988328 21673 hybrid_clock.cc:648] HybridClock initialized: now 1786494883988261 us; error 36 us; skew 500 ppm
I20260812 00:34:43.988678 21673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:43 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494883, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:43.991928 21673 init.cc:377] Logged in from keytab as oryx/127.7.244.125@KRBTEST.COM (short username oryx)
I20260812 00:34:43.993520 21673 webserver.cc:460] Webserver started at http://127.7.244.125:40809/ using document root <none> and password file <none>
I20260812 00:34:43.994277 21673 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:43.994390 21673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:43.994702 21673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:43.998235 21673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data/instance:
uuid: "5ecdfb3b4e0f4487873207bd5c590384"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "1218a6ca9afc4959bcbfc20371f57e37"
server_key_iv: "fbb4df0e11f168417367a9e6a7163dfd"
server_key_version: "encryptionkey@0"
I20260812 00:34:43.999146 21673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal/instance:
uuid: "5ecdfb3b4e0f4487873207bd5c590384"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "1218a6ca9afc4959bcbfc20371f57e37"
server_key_iv: "fbb4df0e11f168417367a9e6a7163dfd"
server_key_version: "encryptionkey@0"
I20260812 00:34:44.001475 21417 heartbeater.cc:503] Master 127.7.244.126:33751 was elected leader, sending a full tablet report...
I20260812 00:34:44.005683 21673 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:34:44.009711 21700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:44.011288 21673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:34:44.011482 21673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
uuid: "5ecdfb3b4e0f4487873207bd5c590384"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "1218a6ca9afc4959bcbfc20371f57e37"
server_key_iv: "fbb4df0e11f168417367a9e6a7163dfd"
server_key_version: "encryptionkey@0"
I20260812 00:34:44.011721 21673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:44.023813 21673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:44.028951 21673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:44.029214 21673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:44.048178 21673 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.125:40257
I20260812 00:34:44.048197 21807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.125:40257 every 8 connection(s)
I20260812 00:34:44.049881 21673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
I20260812 00:34:44.054257 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:44.059731 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21673
I20260812 00:34:44.061533 21810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494883, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.125@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.125@KRBTEST.COM
I20260812 00:34:44.083873 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.064527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46458 (local address 127.7.244.126:33751)
0812 00:34:44.065067 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.065073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.065875 (+ 802us) server_negotiation.cc:407] Connection header received
0812 00:34:44.066099 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.066104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.066200 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.066315 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.068088 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.069274 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.071527 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.072163 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.072308 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.076703 (+ 4395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.076709 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.076713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.076759 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.080149 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.081118 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.081122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.081124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.081258 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.081879 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.081885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.081888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.082301 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.082452 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.083247 (+ 795us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.083505 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":252,"threads_started":1}
I20260812 00:34:44.084998 21818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.064567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:46458)
0812 00:34:44.065694 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.065739 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.065958 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.066430 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.066437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.067305 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.067889 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.067915 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.069470 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.069474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.071340 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.071358 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.072432 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.072438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.072611 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.073414 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.073449 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.076419 (+ 2970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.080334 (+ 3915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.080347 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.080358 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.080980 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.081428 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.081433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.081436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.081750 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.082455 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.082462 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.082846 (+ 384us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.083151 (+ 305us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.083718 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":4,"spinlock_wait_cycles":19328,"thread_start_us":714,"threads_started":1}
I20260812 00:34:44.085016 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.062226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.7.244.125:40257)
0812 00:34:44.063070 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.063085 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.063139 (+ 54us) server_negotiation.cc:407] Connection header received
0812 00:34:44.065366 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.065394 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.065826 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.066207 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.068095 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.069756 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.071522 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.072451 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.072605 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.077254 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.077268 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.077287 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.077409 (+ 122us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.080778 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.081618 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.081625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.081627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.081763 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.082169 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.082174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.082177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.082863 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.083098 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.083533 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.083861 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":594,"spinlock_wait_cycles":13184,"thread_start_us":347,"threads_started":1}
I20260812 00:34:44.087042 21810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494883, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.125@KRBTEST.COM for oryx/127.7.244.125@KRBTEST.COM
I20260812 00:34:44.100083 21818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.087835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.0.0.1:34268)
0812 00:34:44.087996 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.088016 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.088145 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.088468 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.088473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.088848 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.088986 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.088995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.090001 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.090005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.090847 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.090860 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.091510 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.091515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.091698 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.092282 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.092301 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.094522 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.097719 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.097724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.097728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.098115 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.098473 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.098476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.098478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.098611 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.099131 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.099136 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.099289 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.099422 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.099716 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 00:34:44.100272 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.087886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.7.244.125:40257)
0812 00:34:44.088025 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.088029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.088079 (+ 50us) server_negotiation.cc:407] Connection header received
0812 00:34:44.088253 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.088258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.088331 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.088433 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.089113 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.089862 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.090978 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.091373 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.091483 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.094731 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.094736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.094739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.094774 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.097564 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.098235 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.098240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.098243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.098367 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.098720 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.098724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.098726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.099022 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.099153 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.099527 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.099900 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":48}
I20260812 00:34:44.105253 21810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Bootstrap starting.
I20260812 00:34:44.108631 21810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:44.109792 21810 log.cc:824] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:44.112790 21810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: No bootstrap required, opened a new log
I20260812 00:34:44.120527 21810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } }
I20260812 00:34:44.120893 21810 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:44.120968 21810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ecdfb3b4e0f4487873207bd5c590384, State: Initialized, Role: FOLLOWER
I20260812 00:34:44.121577 21810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } }
I20260812 00:34:44.122465 21824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } } }
I20260812 00:34:44.122792 21824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:34:44.123451 21810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 00:34:44.132622 21843 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:34:44.132802 21843 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:34:44.136018 21673 master_runner.cc:428] Detected that this master 5ecdfb3b4e0f4487873207bd5c590384 is joining an existing cluster
I20260812 00:34:44.136148 21673 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.244.125:40257
I20260812 00:34:44.168890 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.140939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34276 (local address 127.7.244.125:40257)
0812 00:34:44.141128 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.141133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.153996 (+ 12863us) server_negotiation.cc:407] Connection header received
0812 00:34:44.154659 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.154666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.154744 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.154868 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.156361 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.157361 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.158801 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.159562 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.159703 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.162261 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.162268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.162273 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.162305 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.165720 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.166598 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.166602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.166605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.166747 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.167137 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.167142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.167145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.167578 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.167707 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.168120 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.168493 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":68}
I20260812 00:34:44.168900 21855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.141474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.0.0.1:34276)
0812 00:34:44.153861 (+ 12387us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.153881 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.154023 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.155046 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.155051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.155837 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.156206 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.156219 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.157539 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.157543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.158669 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.158681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.159775 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.159783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.159987 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.160876 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.160900 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.162054 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.165920 (+ 3866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.165926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.165930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.166452 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.166861 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.166889 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.166893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.167034 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.167686 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.167690 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.167838 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.168002 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.168329 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12229,"mutex_wait_us":47,"spinlock_wait_cycles":9856,"thread_start_us":12015,"threads_started":1}
I20260812 00:34:44.169726 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.148996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46470 (local address 127.7.244.126:33751)
0812 00:34:44.149253 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.149261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.153528 (+ 4267us) server_negotiation.cc:407] Connection header received
0812 00:34:44.154193 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.154202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.154305 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.154438 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.155773 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.156727 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.158918 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.159581 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.159711 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.162046 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.162052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.162055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.162091 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.165732 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.167213 (+ 1481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.167219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.167222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.167404 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.167890 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.167896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.167899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.168272 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.168464 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.169094 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.169392 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260812 00:34:44.170678 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.140832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:46470)
0812 00:34:44.153347 (+ 12515us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.153381 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.154023 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.154494 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.154499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.155232 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.155600 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.155612 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.156916 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.156922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.158758 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.158773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.159963 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.159968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.160092 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.160780 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.160803 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.161818 (+ 1015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.166607 (+ 4789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.166611 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.166614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.167088 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.167535 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.167540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.167543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.167743 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.168527 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.168532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.168749 (+ 217us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.169907 (+ 1158us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.170250 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12286,"mutex_wait_us":30,"spinlock_wait_cycles":22272,"thread_start_us":12714,"threads_started":1}
WARNING: no policy specified for oryx/127.7.244.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.124@KRBTEST.COM" created.
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.126@KRBTEST.COM for oryx/127.7.244.125@KRBTEST.COM
I20260812 00:34:44.196501 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.177899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34288 (local address 127.7.244.125:40257)
0812 00:34:44.178110 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.178117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.178733 (+ 616us) server_negotiation.cc:407] Connection header received
0812 00:34:44.179114 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.179120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.179216 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.179344 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.180675 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.181733 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.183776 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.184376 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.184506 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.189813 (+ 5307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.189821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.189826 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.189878 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.193198 (+ 3320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.194156 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.194162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.194166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.194352 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.194817 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.194822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.194825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.195115 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.195280 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.195819 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.196097 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 00:34:44.196700 21857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.177889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.0.0.1:34288)
0812 00:34:44.178567 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.178614 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.178845 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.179412 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.179420 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.180194 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.180495 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.180508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.181915 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.181922 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.183585 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.183606 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.184767 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.184774 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.185090 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.186189 (+ 1099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.186214 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.189520 (+ 3306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.193431 (+ 3911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.193452 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.193462 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.194001 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.194494 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.194498 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.194501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.194679 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.195280 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.195291 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.195475 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.195663 (+ 188us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.196097 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":540,"threads_started":1}
I20260812 00:34:44.198854 20799 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.244.125:40257: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384"
member_type: NON_VOTER
last_known_addr {
host: "127.7.244.125"
port: 40257
}
attrs {
promote: true
}
}
dest_uuid: "57280ef1740948289e04fb0a6dd73669"
cas_config_opid_index: -1
I20260812 00:34:44.199895 20799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } }
I20260812 00:34:44.210153 21863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5ecdfb3b4e0f4487873207bd5c590384 (127.7.244.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } }
I20260812 00:34:44.212719 21745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.124.keytab.
Entry for principal oryx/127.7.244.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.124.keytab.
I20260812 00:34:44.216075 21863 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.244.125:40257
I20260812 00:34:44.216729 21745 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Refusing update from remote peer 57280ef1740948289e04fb0a6dd73669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 00:34:44.216879 21863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } }
I20260812 00:34:44.217105 21863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:44.218042 21673 master.cc:537] Master@127.7.244.125:40257 shutting down...
I20260812 00:34:44.219038 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.124:42903
--webserver_interface=127.7.244.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.124.keytab
--principal=oryx/127.7.244.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:33751,127.7.244.125:40257,127.7.244.124:42903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:34:44.221758 21864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 00:34:44.226465 20767 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 00:34:44.229043 20767 consensus_peers.cc:487] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 -> Peer 5ecdfb3b4e0f4487873207bd5c590384 (127.7.244.125:40257): Couldn't send request to peer 5ecdfb3b4e0f4487873207bd5c590384. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 00:34:44.230443 21673 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:44.230906 21673 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:44.231112 21673 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: stopping tablet replica
I20260812 00:34:44.237624 21673 master.cc:552] Master@127.7.244.125:40257 shutdown complete.
I20260812 00:34:44.237807 21673 master_runner.cc:305] Clearing existing system tablet
I20260812 00:34:44.238258 21673 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:44.238368 21673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:44.241014 21673 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 00:34:44.242535 21872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:44.242897 21673 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:44.243165 21673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:34:44.243257 21673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
uuid: "5ecdfb3b4e0f4487873207bd5c590384"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "1218a6ca9afc4959bcbfc20371f57e37"
server_key_iv: "fbb4df0e11f168417367a9e6a7163dfd"
server_key_version: "encryptionkey@0"
I20260812 00:34:44.248128 21673 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:44.257897 21673 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 00:34:44.258172 21673 log.cc:1196] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 00:34:44.258724 21673 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Deleting consensus metadata
I20260812 00:34:44.259052 21673 master_runner.cc:315] Copying system tablet from 127.7.244.126:33751
I20260812 00:34:44.268183 21673 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: tablet copy: Beginning tablet copy session from remote peer at address 127.7.244.126:33751
I20260812 00:34:44.281747 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.269093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46484 (local address 127.7.244.126:33751)
0812 00:34:44.269339 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.269347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.269855 (+ 508us) server_negotiation.cc:407] Connection header received
0812 00:34:44.270053 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.270059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.270133 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.270218 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.271180 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.271938 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.273166 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.273710 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.273828 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.275781 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.275787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.275790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.275826 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.278898 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.279632 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.279638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.279640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.279776 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.280189 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.280194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.280196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.280523 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.280665 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.281092 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.281364 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:34:44.281780 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.269027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:46484)
0812 00:34:44.269712 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.269736 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.269906 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.270279 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.270283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.270737 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.270987 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.270997 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.272100 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.272105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.273017 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.273031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.273887 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.273893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.274083 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.274727 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.274750 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.275552 (+ 802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.279070 (+ 3518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.279076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.279081 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.279493 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.279902 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.279908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.279912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.280071 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.280648 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.280653 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.280825 (+ 172us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.280951 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.281359 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":456,"threads_started":1}
I20260812 00:34:44.282963 20835 tablet_copy_service.cc:140] P 57280ef1740948289e04fb0a6dd73669: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5ecdfb3b4e0f4487873207bd5c590384 ({username='oryx', principal='oryx/127.7.244.125@KRBTEST.COM'} at 127.0.0.1:46484)
I20260812 00:34:44.283243 20835 tablet_copy_service.cc:161] P 57280ef1740948289e04fb0a6dd73669: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5ecdfb3b4e0f4487873207bd5c590384 at {username='oryx', principal='oryx/127.7.244.125@KRBTEST.COM'} at 127.0.0.1:46484: session id = 5ecdfb3b4e0f4487873207bd5c590384-00000000000000000000000000000000
I20260812 00:34:44.286651 20835 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 00:34:44.288803 21673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 00:34:44.293920 21673 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: tablet copy: Starting download of 0 data blocks...
I20260812 00:34:44.294274 21673 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: tablet copy: Starting download of 1 WAL segments...
I20260812 00:34:44.297061 21673 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 00:34:44.300922 20835 tablet_copy_service.cc:342] P 57280ef1740948289e04fb0a6dd73669: Request end of tablet copy session 5ecdfb3b4e0f4487873207bd5c590384-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.244.125@KRBTEST.COM'} at 127.0.0.1:46484
I20260812 00:34:44.301146 20835 tablet_copy_service.cc:433] P 57280ef1740948289e04fb0a6dd73669: ending tablet copy session 5ecdfb3b4e0f4487873207bd5c590384-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5ecdfb3b4e0f4487873207bd5c590384
I20260812 00:34:44.312079 21673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:44.314808 21889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:44.318944 21890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:44.323143 21673 server_base.cc:1034] running on GCE node
W20260812 00:34:44.323745 21893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:44.324156 21673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:44.324798 21673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:44.326043 21673 hybrid_clock.cc:648] HybridClock initialized: now 1786494884325969 us; error 37 us; skew 500 ppm
I20260812 00:34:44.326282 21673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494884, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:44.328137 21673 init.cc:377] Logged in from keytab as oryx/127.7.244.125@KRBTEST.COM (short username oryx)
I20260812 00:34:44.329469 21673 webserver.cc:460] Webserver started at http://127.7.244.125:43621/ using document root <none> and password file <none>
I20260812 00:34:44.329798 21673 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:44.329876 21673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:44.332410 21673 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 00:34:44.334132 21906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:44.334514 21673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 00:34:44.334650 21673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
uuid: "5ecdfb3b4e0f4487873207bd5c590384"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "1218a6ca9afc4959bcbfc20371f57e37"
server_key_iv: "fbb4df0e11f168417367a9e6a7163dfd"
server_key_version: "encryptionkey@0"
I20260812 00:34:44.334805 21673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:44.344040 21673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:44.347558 21673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:44.382472 21673 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.125:40257
I20260812 00:34:44.382508 22004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.125:40257 every 8 connection(s)
I20260812 00:34:44.383466 21673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
I20260812 00:34:44.389788 22005 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:34:44.392928 22005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Bootstrap starting.
I20260812 00:34:44.407382 22005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:34:44.407924 22005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Bootstrap complete.
I20260812 00:34:44.409047 22005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } }
I20260812 00:34:44.409410 22005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5ecdfb3b4e0f4487873207bd5c590384, State: Initialized, Role: LEARNER
I20260812 00:34:44.409708 22005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } }
I20260812 00:34:44.412760 22009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } }
I20260812 00:34:44.413102 22005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:44.413187 22009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: This master's current role is: LEARNER
I20260812 00:34:44.421998 22024 catalog_manager.cc:1259] Loaded cluster ID: a6ce5fba19344004b179acb4e647ef35
I20260812 00:34:44.422108 22024 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: loading cluster ID for follower catalog manager: success
I20260812 00:34:44.424638 22024 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: acquiring CA information for follower catalog manager: success
I20260812 00:34:44.426630 22024 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:34:44.536517 21665 heartbeater.cc:503] Master 127.7.244.126:33751 was elected leader, sending a full tablet report...
I20260812 00:34:44.643489 21857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.637135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.0.0.1:34290)
0812 00:34:44.637339 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.637359 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.637496 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.638297 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.638302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.638334 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:44.638526 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.638535 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.639975 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.639981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.641411 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.641426 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.642629 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.642635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.642881 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.642942 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.643032 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.643184 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 00:34:44.643529 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.637361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34290 (local address 127.7.244.125:40257)
0812 00:34:44.637953 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.637958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.637974 (+ 16us) server_negotiation.cc:407] Connection header received
0812 00:34:44.638038 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.638043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.638112 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.638219 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:44.638666 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.639825 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.641570 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.642442 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.642567 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.642838 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.643081 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.643220 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":341,"threads_started":1}
I20260812 00:34:44.644708 21953 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260812 00:34:44.645341 22009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57280ef1740948289e04fb0a6dd73669. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } }
I20260812 00:34:44.645637 22009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: This master's current role is: LEARNER
W20260812 00:34:44.667119 21867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:44.667484 21867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:44.667626 21867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:44.675222 21867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:44.675400 21867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:44.675487 21867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:44.675566 21867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:44.675630 21867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:44.686306 21867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:33751,127.7.244.125:40257,127.7.244.124:42903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.124.keytab
--principal=oryx/127.7.244.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.124:42903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.244.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:44.687785 21867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:44.689163 21867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:44.701980 22036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:44.701910 22042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:44.701952 22039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:44.703320 21867 server_base.cc:1034] running on GCE node
I20260812 00:34:44.706105 21867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:44.708194 21867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:44.709475 21867 hybrid_clock.cc:648] HybridClock initialized: now 1786494884709381 us; error 94 us; skew 500 ppm
I20260812 00:34:44.709925 21867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494884, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:44.714321 21867 init.cc:377] Logged in from keytab as oryx/127.7.244.124@KRBTEST.COM (short username oryx)
I20260812 00:34:44.716243 21867 webserver.cc:460] Webserver started at http://127.7.244.124:44191/ using document root <none> and password file <none>
I20260812 00:34:44.717043 21867 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:44.717170 21867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:44.717558 21867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:44.721163 21867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data/instance:
uuid: "901445618aa9460191789ee586194f48"
format_stamp: "Formatted at 2026-08-12 00:34:44 on dist-test-slave-gjw7"
server_key: "c2ad25c8fc87a890b8c4ddb45b6958d8"
server_key_iv: "eeb981a9e395c234d8afd4a3158c4559"
server_key_version: "encryptionkey@0"
I20260812 00:34:44.722342 21867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal/instance:
uuid: "901445618aa9460191789ee586194f48"
format_stamp: "Formatted at 2026-08-12 00:34:44 on dist-test-slave-gjw7"
server_key: "c2ad25c8fc87a890b8c4ddb45b6958d8"
server_key_iv: "eeb981a9e395c234d8afd4a3158c4559"
server_key_version: "encryptionkey@0"
I20260812 00:34:44.728690 21867 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 00:34:44.732731 22057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:44.734391 21867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:34:44.734578 21867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
uuid: "901445618aa9460191789ee586194f48"
format_stamp: "Formatted at 2026-08-12 00:34:44 on dist-test-slave-gjw7"
server_key: "c2ad25c8fc87a890b8c4ddb45b6958d8"
server_key_iv: "eeb981a9e395c234d8afd4a3158c4559"
server_key_version: "encryptionkey@0"
I20260812 00:34:44.734839 21867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:44.765950 21867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:44.770988 21867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:44.771296 21867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:44.791049 21867 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.124:42903
I20260812 00:34:44.791087 22141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.124:42903 every 8 connection(s)
I20260812 00:34:44.792618 21867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
I20260812 00:34:44.795894 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21867
I20260812 00:34:44.797072 22142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:44.808256 22142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.124@KRBTEST.COM
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494884, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.124@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:44.824065 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.798604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.7.244.124:42903)
0812 00:34:44.799568 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.799586 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.799645 (+ 59us) server_negotiation.cc:407] Connection header received
0812 00:34:44.801629 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.801667 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.802287 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.802831 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.804479 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.806661 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.808067 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.809142 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.809381 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.814873 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.814885 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.814902 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.814952 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.818703 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.819626 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.819636 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.819649 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.819859 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.820335 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.820341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.820344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.820973 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.821454 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.822091 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.822589 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":375,"threads_started":1}
I20260812 00:34:44.828441 22150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.811730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.7.244.126:33751)
0812 00:34:44.812226 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.812232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.812788 (+ 556us) server_negotiation.cc:407] Connection header received
0812 00:34:44.812981 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.812987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.813064 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.813160 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.814471 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.815609 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.817720 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.818180 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.818296 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.822265 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.822272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.822276 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.822322 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.825453 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.826257 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.826262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.826265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.826413 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.826823 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.826828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.826830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.827164 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.827310 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.827810 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.828056 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":254,"threads_started":1}
I20260812 00:34:44.828480 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.811695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:46494)
0812 00:34:44.812595 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.812634 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.812822 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.813250 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.813258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.813995 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.814309 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.814319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.815748 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.815752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.817546 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.817563 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.818409 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.818417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.818613 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.819277 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.819296 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.822032 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.825606 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.825620 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.825628 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.826111 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.826530 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.826536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.826540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.826701 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.827296 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.827304 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.827493 (+ 189us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.827658 (+ 165us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.828046 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":516,"threads_started":1}
I20260812 00:34:44.830200 22142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494884, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.124@KRBTEST.COM for oryx/127.7.244.125@KRBTEST.COM
I20260812 00:34:44.843710 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.831063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34298 (local address 127.7.244.125:40257)
0812 00:34:44.831272 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.831278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.831297 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:34:44.831448 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.831453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.831518 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.831602 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.832334 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.833054 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.834399 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.834821 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.834940 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.838246 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.838252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.838255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.838293 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.840881 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.841674 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.841679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.841681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.841834 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.842274 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.842278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.842280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.842577 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.842717 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.843147 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.843390 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 00:34:44.843703 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.830977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.0.0.1:34298)
0812 00:34:44.831183 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.831204 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.831325 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.831641 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.831645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.832019 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.832172 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.832180 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.833191 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.833196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.834245 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.834259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.834980 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.834986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.835196 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.835801 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.835819 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.837997 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.841035 (+ 3038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.841041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.841045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.841541 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.841981 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.841984 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.841987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.842145 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.842699 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.842704 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.842875 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.842994 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.843338 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260812 00:34:44.844951 22142 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494884, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.124@KRBTEST.COM for oryx/127.7.244.124@KRBTEST.COM
I20260812 00:34:44.858646 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.845708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42903 (local address 127.0.0.1:43262)
0812 00:34:44.845912 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.845930 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.846027 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.846288 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.846291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.846652 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.846815 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.846823 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.847825 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.847833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.848858 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.848874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.849655 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.849660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.849909 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.850682 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.850701 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.852920 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.855936 (+ 3016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.855941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.855944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.856367 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.856830 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.856835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.856838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.857030 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.857753 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.857758 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.857937 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.858063 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.858299 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 00:34:44.858882 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.845879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43262 (local address 127.7.244.124:42903)
0812 00:34:44.846019 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.846023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.846040 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:34:44.846120 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.846123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.846180 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.846255 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.846943 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.847666 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.849014 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.849492 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.849588 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.853163 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.853169 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.853173 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.853210 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.855739 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.856539 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.856546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.856548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.856697 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.857158 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.857162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.857164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.857618 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.857748 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.858158 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.858500 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":50}
I20260812 00:34:44.863842 22142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Bootstrap starting.
I20260812 00:34:44.867333 22142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:44.868537 22142 log.cc:824] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:44.872138 22142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: No bootstrap required, opened a new log
I20260812 00:34:44.880759 22142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } }
I20260812 00:34:44.881179 22142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:44.881352 22142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901445618aa9460191789ee586194f48, State: Initialized, Role: FOLLOWER
I20260812 00:34:44.881989 22142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } }
I20260812 00:34:44.883001 22158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } } }
I20260812 00:34:44.883342 22158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:34:44.884087 22142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 00:34:44.893975 22180 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:34:44.894166 22180 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:34:44.899780 21867 master_runner.cc:428] Detected that this master 901445618aa9460191789ee586194f48 is joining an existing cluster
I20260812 00:34:44.899930 21867 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.244.124:42903
I20260812 00:34:44.919490 22158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:44.919854 22158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } }
I20260812 00:34:44.921924 22158 leader_election.cc:288] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751), 5ecdfb3b4e0f4487873207bd5c590384 (127.7.244.125:40257)
I20260812 00:34:44.923193 21953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "901445618aa9460191789ee586194f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ecdfb3b4e0f4487873207bd5c590384" is_pre_election: true
I20260812 00:34:44.924098 20816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "901445618aa9460191789ee586194f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57280ef1740948289e04fb0a6dd73669" is_pre_election: true
I20260812 00:34:44.925570 22150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.904244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.7.244.126:33751)
0812 00:34:44.904472 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.904479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.905379 (+ 900us) server_negotiation.cc:407] Connection header received
0812 00:34:44.905696 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.905703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.905817 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.905904 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.907691 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.908949 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.911025 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.911710 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.911825 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.914975 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.914984 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.914988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.915035 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.918974 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.922905 (+ 3931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.922912 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.922915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.923118 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.923666 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.923671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.923674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.924056 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.924240 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.924902 (+ 662us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.925176 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:34:44.925887 22062 leader_election.cc:302] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 901445618aa9460191789ee586194f48; no voters: 57280ef1740948289e04fb0a6dd73669, 5ecdfb3b4e0f4487873207bd5c590384
I20260812 00:34:44.926703 22191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.904294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:46506)
0812 00:34:44.905139 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.905211 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.905514 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.906165 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.906170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.906966 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.907472 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.907488 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.909216 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.909222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.910847 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.910863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.912152 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.912159 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.912339 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.913183 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.913206 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.914698 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.922212 (+ 7514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.922219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.922223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.922759 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.923301 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.923305 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.923309 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.923508 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.924346 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.924352 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.924572 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.925827 (+ 1255us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.926226 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":426,"threads_started":1}
I20260812 00:34:44.927157 22158 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 00:34:44.930828 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20910
I20260812 00:34:44.933194 22025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.904793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34306 (local address 127.7.244.125:40257)
0812 00:34:44.904981 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.904987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.910348 (+ 5361us) server_negotiation.cc:407] Connection header received
0812 00:34:44.910500 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.910506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.910596 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.910701 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.912346 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.913505 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.916177 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.916884 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.916998 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.920376 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.920383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.920387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.920436 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.924722 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.925596 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.925602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.925605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.925793 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.926274 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.926280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.926284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.926631 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.926785 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.928582 (+ 1797us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.928876 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:34:44.934419 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.904300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.0.0.1:34306)
0812 00:34:44.910184 (+ 5884us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.910210 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.910373 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.910914 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.910920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.911734 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.912144 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.912154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.914140 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.914155 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.915985 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.916003 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.917879 (+ 1876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.917890 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.918141 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.918866 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.918896 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.920124 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.924908 (+ 4784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.924913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.924916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.925427 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.925945 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.925950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.925953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.926137 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.926875 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.926880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.928272 (+ 1392us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.933450 (+ 5178us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.933892 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5709,"mutex_wait_us":93,"thread_start_us":12515,"threads_started":1}
I20260812 00:34:44.940954 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.905643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43266 (local address 127.7.244.124:42903)
0812 00:34:44.905816 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.905822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.916924 (+ 11102us) server_negotiation.cc:407] Connection header received
0812 00:34:44.917444 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.917451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.917534 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.917637 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.919113 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.920284 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.928363 (+ 8079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.929046 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.929151 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.932045 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.932052 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.932055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.932089 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.936314 (+ 4225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.938028 (+ 1714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.938035 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.938038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.938242 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.938781 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.938786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.938789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.939238 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.939484 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.940154 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.940504 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.126@KRBTEST.COM for oryx/127.7.244.124@KRBTEST.COM
I20260812 00:34:44.954428 22192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.905324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42903 (local address 127.0.0.1:43266)
0812 00:34:44.916824 (+ 11500us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.916846 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.917277 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.917685 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.917688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.918532 (+ 844us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.918933 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.918947 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.921069 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.921075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.928181 (+ 7106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.928200 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.929518 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.929526 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.929689 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.930615 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.930642 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.931852 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.937122 (+ 5270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.937128 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.937132 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.937753 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.938405 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.938409 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.938413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.938629 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.939569 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.939574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.939889 (+ 315us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.953505 (+ 13616us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.953877 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11340,"thread_start_us":4442,"threads_started":1}
I20260812 00:34:44.960397 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.934647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43280 (local address 127.7.244.124:42903)
0812 00:34:44.941590 (+ 6943us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.941596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.941618 (+ 22us) server_negotiation.cc:407] Connection header received
0812 00:34:44.941730 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.941735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.941850 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.943116 (+ 1266us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.943326 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.944548 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.946473 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.947175 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.952395 (+ 5220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.952642 (+ 247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.952649 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.952655 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.952688 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.957282 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.958073 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.958079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.958082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.958247 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.958756 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.958761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.958764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.959190 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.959348 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.959902 (+ 554us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.960125 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6835,"thread_start_us":16743,"threads_started":1}
I20260812 00:34:44.960521 21857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.933741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42903 (local address 127.0.0.1:43280)
0812 00:34:44.934137 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:44.934156 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:44.934266 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:44.942078 (+ 7812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:44.942085 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:44.942685 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:44.942941 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.942959 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.944760 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.944767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.946215 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:44.946236 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.947422 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.947429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:44.947616 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.948423 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:44.948450 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:44.952153 (+ 3703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:44.957479 (+ 5326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.957484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.957487 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.957930 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.958402 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:44.958408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:44.958412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:44.958584 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:44.959362 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:44.959369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:44.959570 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 00:34:44.959760 (+ 190us) client_negotiation.cc:240] Negotiation successful
0812 00:34:44.960113 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260812 00:34:44.962056 20799 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.244.124:42903: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "901445618aa9460191789ee586194f48"
member_type: NON_VOTER
last_known_addr {
host: "127.7.244.124"
port: 42903
}
attrs {
promote: true
}
}
dest_uuid: "57280ef1740948289e04fb0a6dd73669"
cas_config_opid_index: 5
I20260812 00:34:44.963092 20799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } }
I20260812 00:34:44.965085 21863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 901445618aa9460191789ee586194f48 (127.7.244.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:44.966315 22102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:44.967474 21953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 LEARNER]: Refusing update from remote peer 57280ef1740948289e04fb0a6dd73669: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 00:34:44.969027 21863 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.244.124:42903
I20260812 00:34:44.970048 22102 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Refusing update from remote peer 57280ef1740948289e04fb0a6dd73669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
Aug 12 00:34:44 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.65@KRBTEST.COM
I20260812 00:34:44.982134 22198 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 00:34:44.983520 21863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:44.983011 22199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:44.984338 22199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:44.992331 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:44.932466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.7.244.65:44493)
0812 00:34:44.957420 (+ 24954us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:44.957431 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:44.957467 (+ 36us) server_negotiation.cc:407] Connection header received
0812 00:34:44.957560 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:44.957569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:44.957881 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:44.958171 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:44.960946 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.962722 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.964627 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:44.965627 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:44.966457 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:44.979080 (+ 12623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:44.979111 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:44.979138 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:44.979227 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:44.986828 (+ 7601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.988107 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.988120 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.988125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.988367 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:44.988956 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:44.988961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:44.988965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:44.989524 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:44.991296 (+ 1772us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:44.991498 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 00:34:44.991836 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24723,"thread_start_us":24544,"threads_started":1}
I20260812 00:34:44.994341 21151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:44.996034 21953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 901445618aa9460191789ee586194f48 (127.7.244.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.001356 22009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.001704 22009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: This master's current role is: LEARNER
I20260812 00:34:45.002215 22200 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: attempting to promote NON_VOTER 5ecdfb3b4e0f4487873207bd5c590384 to VOTER
I20260812 00:34:45.002919 22200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } }
I20260812 00:34:45.005743 22101 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Refusing update from remote peer 57280ef1740948289e04fb0a6dd73669: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 00:34:45.006822 22199 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:45.007107 21953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 LEARNER]: Refusing update from remote peer 57280ef1740948289e04fb0a6dd73669: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 00:34:45.007740 21863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 00:34:45.010462 21863 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5ecdfb3b4e0f4487873207bd5c590384 (127.7.244.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.013715 21867 master.cc:537] Master@127.7.244.124:42903 shutting down...
W20260812 00:34:45.023643 22067 tablet.cc:2369] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:45.026345 21863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.026722 21863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:45.026994 22198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.027290 22198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:45.028065 21953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5ecdfb3b4e0f4487873207bd5c590384 (127.7.244.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.032832 22009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.033156 22009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:34:45.035802 22101 raft_consensus.cc:858] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } }
W20260812 00:34:45.036166 22101 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } }
I20260812 00:34:45.045269 22101 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5ecdfb3b4e0f4487873207bd5c590384 (127.7.244.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } }
I20260812 00:34:45.050511 22218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } }
I20260812 00:34:45.051014 22218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 00:34:45.051512 22158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57280ef1740948289e04fb0a6dd73669. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } }
I20260812 00:34:45.051748 22158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 00:34:45.051985 22102 raft_consensus.cc:1467] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260812 00:34:45.052148 22102 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 NON_PARTICIPANT]: Deduplicated request from leader. Original: 0.0->[1.1-1.6] Dedup: 1.5->[1.6-1.6]
I20260812 00:34:45.052768 21863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.053069 21863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:45.052747 22102 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 901445618aa9460191789ee586194f48 (127.7.244.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.056558 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } }
I20260812 00:34:45.056893 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 00:34:45.058185 22214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: NON_VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: true } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.058461 22214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: LEARNER
I20260812 00:34:45.062314 21867 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 LEARNER]: Raft consensus shutting down.
W20260812 00:34:45.062839 20768 consensus_peers.cc:487] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 -> Peer 901445618aa9460191789ee586194f48 (127.7.244.124:42903): Couldn't send request to peer 901445618aa9460191789ee586194f48. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 00:34:45.062892 21867 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 LEARNER]: Raft consensus is shut down!
I20260812 00:34:45.063082 21867 tablet_replica.cc:330] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: stopping tablet replica
I20260812 00:34:45.071698 21867 master.cc:552] Master@127.7.244.124:42903 shutdown complete.
I20260812 00:34:45.071803 21867 master_runner.cc:305] Clearing existing system tablet
I20260812 00:34:45.072227 21867 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:45.072331 21867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:45.081547 21867 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.005s
I20260812 00:34:45.084060 22224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:45.084582 21867 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:45.085078 21867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 00:34:45.085315 21867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
uuid: "901445618aa9460191789ee586194f48"
format_stamp: "Formatted at 2026-08-12 00:34:44 on dist-test-slave-gjw7"
server_key: "c2ad25c8fc87a890b8c4ddb45b6958d8"
server_key_iv: "eeb981a9e395c234d8afd4a3158c4559"
server_key_version: "encryptionkey@0"
I20260812 00:34:45.090724 21867 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:45.098973 21867 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 00:34:45.099339 21867 log.cc:1196] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 00:34:45.100155 21867 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Deleting consensus metadata
I20260812 00:34:45.100553 21867 master_runner.cc:315] Copying system tablet from 127.7.244.126:33751
I20260812 00:34:45.103693 21867 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: tablet copy: Beginning tablet copy session from remote peer at address 127.7.244.126:33751
I20260812 00:34:45.114040 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20910
I20260812 00:34:45.138411 22150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.104953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.7.244.126:33751)
0812 00:34:45.105224 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:45.105230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:45.105919 (+ 689us) server_negotiation.cc:407] Connection header received
0812 00:34:45.106261 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:45.106268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:45.106359 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:45.106437 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:45.107784 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.108991 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.111030 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.111723 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.111852 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.131556 (+ 19704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:45.131565 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:45.131570 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:45.131634 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:45.134819 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.135821 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.135828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.135831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.136120 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.136645 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.136649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.136652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.136969 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:45.137090 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:45.137676 (+ 586us) server_negotiation.cc:299] Negotiation successful
0812 00:34:45.137978 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260812 00:34:45.139643 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.104837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:46518)
0812 00:34:45.105773 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:45.105795 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:45.106066 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:45.106670 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:45.106675 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:45.107303 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:45.107573 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.107587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.109209 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.109215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.110824 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.110842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.111941 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.111948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.112160 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.112933 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:45.112956 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:45.114023 (+ 1067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:45.135102 (+ 21079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:45.135108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:45.135111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:45.135672 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:45.136290 (+ 618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:45.136294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:45.136297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:45.136527 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:45.137172 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:45.137180 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:45.137380 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 00:34:45.138666 (+ 1286us) client_negotiation.cc:240] Negotiation successful
0812 00:34:45.139089 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":630,"threads_started":1}
I20260812 00:34:45.140357 20835 tablet_copy_service.cc:140] P 57280ef1740948289e04fb0a6dd73669: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 901445618aa9460191789ee586194f48 ({username='oryx', principal='oryx/127.7.244.124@KRBTEST.COM'} at 127.0.0.1:46518)
I20260812 00:34:45.140547 20835 tablet_copy_service.cc:161] P 57280ef1740948289e04fb0a6dd73669: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 901445618aa9460191789ee586194f48 at {username='oryx', principal='oryx/127.7.244.124@KRBTEST.COM'} at 127.0.0.1:46518: session id = 901445618aa9460191789ee586194f48-00000000000000000000000000000000
I20260812 00:34:45.144116 20835 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 00:34:45.146369 21867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 00:34:45.153986 21867 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: tablet copy: Starting download of 0 data blocks...
I20260812 00:34:45.154382 21867 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: tablet copy: Starting download of 1 WAL segments...
I20260812 00:34:45.157791 21867 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 00:34:45.160988 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:44493
--local_ip_for_outbound_sockets=127.7.244.65
--tserver_master_addrs=127.7.244.126:33751
--webserver_port=43321
--webserver_interface=127.7.244.65
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab
--principal=oryx/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257,127.7.244.124:42903 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:34:45.163126 20835 tablet_copy_service.cc:342] P 57280ef1740948289e04fb0a6dd73669: Request end of tablet copy session 901445618aa9460191789ee586194f48-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.244.124@KRBTEST.COM'} at 127.0.0.1:46518
I20260812 00:34:45.163303 20835 tablet_copy_service.cc:433] P 57280ef1740948289e04fb0a6dd73669: ending tablet copy session 901445618aa9460191789ee586194f48-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 901445618aa9460191789ee586194f48
I20260812 00:34:45.176272 21867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:45.179211 22239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:45.179642 22240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:45.187551 22246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:45.187729 21867 server_base.cc:1034] running on GCE node
I20260812 00:34:45.188073 21867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:45.188755 21867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:45.189981 21867 hybrid_clock.cc:648] HybridClock initialized: now 1786494885189926 us; error 43 us; skew 500 ppm
I20260812 00:34:45.190263 21867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:45 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494885, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:45.192306 21867 init.cc:377] Logged in from keytab as oryx/127.7.244.124@KRBTEST.COM (short username oryx)
I20260812 00:34:45.193713 21867 webserver.cc:460] Webserver started at http://127.7.244.124:35065/ using document root <none> and password file <none>
I20260812 00:34:45.194032 21867 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:45.194128 21867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:45.196727 21867 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 00:34:45.198789 22265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:45.199229 21867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 00:34:45.199368 21867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
uuid: "901445618aa9460191789ee586194f48"
format_stamp: "Formatted at 2026-08-12 00:34:44 on dist-test-slave-gjw7"
server_key: "c2ad25c8fc87a890b8c4ddb45b6958d8"
server_key_iv: "eeb981a9e395c234d8afd4a3158c4559"
server_key_version: "encryptionkey@0"
I20260812 00:34:45.199553 21867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:45.226168 21867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:45.229993 21867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:45.272776 21867 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.124:42903
I20260812 00:34:45.272809 22354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.124:42903 every 8 connection(s)
I20260812 00:34:45.273959 21867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
I20260812 00:34:45.279804 22355 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:34:45.283969 22355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Bootstrap starting.
I20260812 00:34:45.302361 22355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:34:45.302932 22355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: Bootstrap complete.
I20260812 00:34:45.304309 22355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } }
I20260812 00:34:45.304642 22355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 901445618aa9460191789ee586194f48, State: Initialized, Role: LEARNER
I20260812 00:34:45.305019 22355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } }
I20260812 00:34:45.306267 22355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:45.305979 22361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.306535 22361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: LEARNER
I20260812 00:34:45.318145 22389 catalog_manager.cc:1259] Loaded cluster ID: a6ce5fba19344004b179acb4e647ef35
I20260812 00:34:45.318297 22389 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: loading cluster ID for follower catalog manager: success
I20260812 00:34:45.322074 22389 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: acquiring CA information for follower catalog manager: success
I20260812 00:34:45.324939 22389 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:34:45.473167 22198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.473604 22198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:45.480427 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.473571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42903 (local address 127.0.0.1:43290)
0812 00:34:45.474115 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:45.474134 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:45.474352 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:45.474933 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:45.474940 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:45.474974 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:45.475259 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.475272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.476665 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.476673 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.478084 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.478102 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.479457 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.479465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.479685 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.479728 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 00:34:45.479835 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 00:34:45.480002 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":576,"threads_started":1}
I20260812 00:34:45.480427 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.473956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.7.244.124:42903)
0812 00:34:45.474574 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:45.474578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:45.474600 (+ 22us) server_negotiation.cc:407] Connection header received
0812 00:34:45.474681 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:45.474684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:45.474763 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:45.474884 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:45.475446 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.476488 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.478279 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.479200 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.479352 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.479658 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:45.479877 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 00:34:45.480033 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":371,"threads_started":1}
I20260812 00:34:45.481849 22318 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260812 00:34:45.482381 22361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57280ef1740948289e04fb0a6dd73669. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: NON_VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: true } } }
I20260812 00:34:45.482684 22361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: LEARNER
I20260812 00:34:45.483258 22199 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: attempting to promote NON_VOTER 901445618aa9460191789ee586194f48 to VOTER
I20260812 00:34:45.483978 22199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } }
I20260812 00:34:45.486080 22318 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 LEARNER]: Refusing update from remote peer 57280ef1740948289e04fb0a6dd73669: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 00:34:45.486178 21953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Refusing update from remote peer 57280ef1740948289e04fb0a6dd73669: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 00:34:45.486871 22198 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 00:34:45.487474 22198 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 00:34:45.490206 22198 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 901445618aa9460191789ee586194f48 (127.7.244.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:45.490921 21953 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 901445618aa9460191789ee586194f48 (127.7.244.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:45.495220 21863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:45.495308 22198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:45.495574 21863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:45.495609 22198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:45.496805 22009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:45.497124 22009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:34:45.503151 22198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:45.503592 22198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:45.504295 22318 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 901445618aa9460191789ee586194f48 (127.7.244.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:45.509085 22361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57280ef1740948289e04fb0a6dd73669" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:45.509475 22361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 00:34:45.582651 22237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:45.582974 22237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:45.583071 22237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:45.590756 22237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:45.590903 22237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:45.591022 22237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:45.602403 22237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab
--principal=oryx/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:44493
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=43321
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257,127.7.244.124:42903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:45.604435 22237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:45.606477 22237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:45.624878 22416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:45.624845 22412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:45.626232 22411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:45.626646 22237 server_base.cc:1034] running on GCE node
I20260812 00:34:45.627369 22237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:45.628531 22237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:45.632128 22237 hybrid_clock.cc:648] HybridClock initialized: now 1786494885632035 us; error 72 us; skew 500 ppm
I20260812 00:34:45.632557 22237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:45 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494885, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:45.636454 22237 init.cc:377] Logged in from keytab as oryx/127.7.244.65@KRBTEST.COM (short username oryx)
I20260812 00:34:45.638710 22237 webserver.cc:460] Webserver started at http://127.7.244.65:43321/ using document root <none> and password file <none>
I20260812 00:34:45.639879 22237 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:45.640077 22237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:45.648530 22237 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 00:34:45.652410 22429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:45.654501 22237 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 00:34:45.654729 22237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "e640fd14b3b1411fa515c1cdbd2d113d"
format_stamp: "Formatted at 2026-08-12 00:34:42 on dist-test-slave-gjw7"
server_key: "90f63719f1ea692617b0c5eee3cf1af0"
server_key_iv: "af971473152d0ab5e65dc79d944149fd"
server_key_version: "encryptionkey@0"
I20260812 00:34:45.656286 22237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:45.678301 22237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:45.682914 22237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:45.683161 22237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:45.684183 22237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:45.685698 22237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:45.685793 22237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:45.685894 22237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:45.685949 22237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:45.727288 22237 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:44493
I20260812 00:34:45.727329 22666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:44493 every 8 connection(s)
I20260812 00:34:45.728950 22237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:45.739073 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 22237
I20260812 00:34:45.739446 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21169
Aug 12 00:34:45 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494885, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.125@KRBTEST.COM
Aug 12 00:34:45 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494885, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
Aug 12 00:34:45 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494885, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.124@KRBTEST.COM
I20260812 00:34:45.759683 22680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.734556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:60381 (local address 127.7.244.126:33751)
0812 00:34:45.735259 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:45.735264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:45.735281 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:34:45.736341 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:45.736347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:45.736430 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:45.736532 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:45.738297 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.739510 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.742405 (+ 2895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.743042 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.743173 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.750967 (+ 7794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:45.750978 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:45.750983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:45.751033 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:45.754802 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.756166 (+ 1364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.756172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.756176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.756372 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.757131 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.757137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.757140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.757501 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:45.757651 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:45.759034 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0812 00:34:45.759292 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":483,"threads_started":1}
I20260812 00:34:45.760895 22678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.734383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:60381)
0812 00:34:45.734923 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:45.734942 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:45.735940 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:45.736715 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:45.736720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:45.737607 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:45.738146 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.738156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.739705 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.739709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.742196 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.742219 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.743526 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.743535 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.743952 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.744964 (+ 1012us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:45.745011 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:45.750387 (+ 5376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:45.754992 (+ 4605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:45.755002 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:45.755022 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:45.755987 (+ 965us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:45.756524 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:45.756529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:45.756533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:45.756869 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:45.757736 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:45.757745 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:45.758382 (+ 637us) client_negotiation.cc:769] Sending connection context
0812 00:34:45.758720 (+ 338us) client_negotiation.cc:240] Negotiation successful
0812 00:34:45.760322 (+ 1602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":512,"spinlock_wait_cycles":82944,"thread_start_us":286,"threads_started":1}
I20260812 00:34:45.761515 22675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.732525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.7.244.65:47573)
0812 00:34:45.733485 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:45.733546 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:45.735948 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:45.736685 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:45.736693 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:45.737380 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:45.738023 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.738044 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.739635 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.739640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.742207 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.742231 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.743605 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.743612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.743963 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.744969 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:45.745006 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:45.750387 (+ 5381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:45.755081 (+ 4694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:45.755085 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:45.755088 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:45.755987 (+ 899us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:45.756524 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:45.756529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:45.756533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:45.756869 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:45.757750 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:45.757753 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:45.758382 (+ 629us) client_negotiation.cc:769] Sending connection context
0812 00:34:45.758720 (+ 338us) client_negotiation.cc:240] Negotiation successful
0812 00:34:45.759313 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":147,"spinlock_wait_cycles":50176,"thread_start_us":531,"threads_started":1}
I20260812 00:34:45.761582 22674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.732002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:47573 (local address 127.7.244.125:40257)
0812 00:34:45.732540 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:45.732547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:45.734877 (+ 2330us) server_negotiation.cc:407] Connection header received
0812 00:34:45.736227 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:45.736234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:45.736321 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:45.736419 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:45.738219 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.739407 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.742417 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.743170 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.743274 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.751016 (+ 7742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:45.751025 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:45.751031 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:45.751084 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:45.754947 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.756166 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.756172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.756176 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.756372 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.757068 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.757073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.757077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.757501 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:45.757650 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:45.760894 (+ 3244us) server_negotiation.cc:299] Negotiation successful
0812 00:34:45.761154 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":664,"threads_started":1}
I20260812 00:34:45.763028 22670 heartbeater.cc:347] Connected to a master server at 127.7.244.125:40257
I20260812 00:34:45.763058 22668 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33751
I20260812 00:34:45.763161 22676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.733025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42903 (local address 127.7.244.65:60059)
0812 00:34:45.733756 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:45.733775 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:45.735935 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:45.736715 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:45.736720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:45.737607 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:45.738101 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.738116 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.739691 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.739696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.742252 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:45.742263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.743776 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.743782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:45.743952 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.745098 (+ 1146us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:45.745117 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:45.750709 (+ 5592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:45.757285 (+ 6576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:45.757315 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:45.757319 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:45.757938 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:45.758525 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:45.758531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:45.758540 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:45.758755 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:45.759457 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:45.759462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:45.759639 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 00:34:45.759785 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 00:34:45.760107 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":1060,"spinlock_wait_cycles":22016,"thread_start_us":425,"threads_started":1}
I20260812 00:34:45.763435 22670 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:45.763437 22668 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:45.764628 22670 heartbeater.cc:511] Master 127.7.244.125:40257 requested a full tablet report, sending...
I20260812 00:34:45.764652 22668 heartbeater.cc:511] Master 127.7.244.126:33751 requested a full tablet report, sending...
I20260812 00:34:45.764817 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.732371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:60059 (local address 127.7.244.124:42903)
0812 00:34:45.732581 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:45.732588 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:45.733869 (+ 1281us) server_negotiation.cc:407] Connection header received
0812 00:34:45.736342 (+ 2473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:45.736348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:45.736466 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:45.736550 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:45.738297 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.739432 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.742416 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.743081 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.743187 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.750967 (+ 7780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:45.750976 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:45.750980 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:45.751033 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:45.757082 (+ 6049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.758171 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.758177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.758181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.758384 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.758931 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.758937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.758940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.759312 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:45.763857 (+ 4545us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:45.764054 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 00:34:45.764366 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:34:45.766481 22667 heartbeater.cc:347] Connected to a master server at 127.7.244.124:42903
I20260812 00:34:45.766639 22667 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:45.766973 20799 ts_manager.cc:194] Re-registered known tserver with Master: e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65:44493)
I20260812 00:34:45.767076 22667 heartbeater.cc:511] Master 127.7.244.124:42903 requested a full tablet report, sending...
I20260812 00:34:45.767428 21935 ts_manager.cc:194] Registered new tserver with Master: e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65:44493)
I20260812 00:34:45.768121 20799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:60381
I20260812 00:34:45.769042 22302 ts_manager.cc:194] Registered new tserver with Master: e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65:44493)
Aug 12 00:34:45 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.66@KRBTEST.COM
I20260812 00:34:45.779580 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:45.740880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46934 (local address 127.7.244.66:45217)
0812 00:34:45.760156 (+ 19276us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:45.760165 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:45.760199 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:34:45.760272 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:45.760281 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:45.760601 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:45.763370 (+ 2769us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:45.763666 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.765238 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.767236 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:45.768129 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:45.768934 (+ 805us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:45.773359 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:45.773379 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:45.773408 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:45.773461 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:45.776458 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.777264 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.777273 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.777275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.777483 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:45.777898 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:45.777903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:45.777905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:45.778277 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:45.778583 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:45.778922 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 00:34:45.779211 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19065,"thread_start_us":19869,"threads_started":1}
I20260812 00:34:45.780957 21410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:45.865980 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21169
I20260812 00:34:45.896436 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:45217
--local_ip_for_outbound_sockets=127.7.244.66
--tserver_master_addrs=127.7.244.126:33751
--webserver_port=45867
--webserver_interface=127.7.244.66
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab
--principal=oryx/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257,127.7.244.124:42903 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:46.253230 22685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:46.253626 22685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:46.253756 22685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:46.261688 22685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:46.261821 22685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:46.261973 22685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:46.272514 22685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab
--principal=oryx/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:45217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=45867
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257,127.7.244.124:42903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:46.274008 22685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:46.275403 22685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:46.289436 22694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:46.289520 22701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:46.289477 22695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:46.290567 22685 server_base.cc:1034] running on GCE node
I20260812 00:34:46.291285 22685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:46.292400 22685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:46.293618 22685 hybrid_clock.cc:648] HybridClock initialized: now 1786494886293565 us; error 51 us; skew 500 ppm
I20260812 00:34:46.293972 22685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:46 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494886, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:46.297420 22685 init.cc:377] Logged in from keytab as oryx/127.7.244.66@KRBTEST.COM (short username oryx)
I20260812 00:34:46.299094 22685 webserver.cc:460] Webserver started at http://127.7.244.66:45867/ using document root <none> and password file <none>
I20260812 00:34:46.299908 22685 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:46.300056 22685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:46.306200 22685 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:34:46.309576 22713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:46.311137 22685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 00:34:46.311340 22685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "94e9e1797474436ab2a79319877c53dd"
format_stamp: "Formatted at 2026-08-12 00:34:42 on dist-test-slave-gjw7"
server_key: "06e97f84fbcc3ef7f2bde60ee57ea3ed"
server_key_iv: "3f219b00e75c1436abfd5355dd713e35"
server_key_version: "encryptionkey@0"
I20260812 00:34:46.312433 22685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:46.349866 22685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:46.354799 22685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:46.355098 22685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:46.356185 22685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:46.357754 22685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:46.357868 22685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:46.357983 22685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:46.358054 22685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:46.395105 22685 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:45217
I20260812 00:34:46.395175 22948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:45217 every 8 connection(s)
I20260812 00:34:46.396754 22685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:46.403657 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 22685
I20260812 00:34:46.404002 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21429
Aug 12 00:34:46 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494886, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.125@KRBTEST.COM
Aug 12 00:34:46 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494886, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
Aug 12 00:34:46 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494886, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.124@KRBTEST.COM
Aug 12 00:34:46 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.67@KRBTEST.COM
I20260812 00:34:46.435091 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:46.400206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:41207 (local address 127.7.244.125:40257)
0812 00:34:46.400743 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:46.400749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:46.402008 (+ 1259us) server_negotiation.cc:407] Connection header received
0812 00:34:46.408743 (+ 6735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:46.408751 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:46.408864 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:46.408973 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:46.410363 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.411630 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.418750 (+ 7120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.419287 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.419399 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:46.426596 (+ 7197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:46.426604 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:46.426610 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:46.426656 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:46.430486 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:46.431877 (+ 1391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:46.431883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:46.431887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:46.432066 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:46.432666 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:46.432672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:46.432675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:46.433081 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:46.433227 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:46.434448 (+ 1221us) server_negotiation.cc:299] Negotiation successful
0812 00:34:46.434704 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":268,"threads_started":1}
I20260812 00:34:46.435091 22954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:46.399880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:52405 (local address 127.7.244.126:33751)
0812 00:34:46.400369 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:46.400376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:46.402064 (+ 1688us) server_negotiation.cc:407] Connection header received
0812 00:34:46.405236 (+ 3172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:46.405245 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:46.405383 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:46.405477 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:46.413487 (+ 8010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.414790 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.420671 (+ 5881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.421391 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.421537 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:46.426967 (+ 5430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:46.426974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:46.426977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:46.427015 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:46.430870 (+ 3855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:46.431968 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:46.431974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:46.431977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:46.432145 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:46.432666 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:46.432672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:46.432675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:46.433071 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:46.433208 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:46.434462 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0812 00:34:46.434702 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":484,"threads_started":1}
I20260812 00:34:46.436726 22965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:46.402684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:58047 (local address 127.7.244.124:42903)
0812 00:34:46.418129 (+ 15445us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:46.418135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:46.418156 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:34:46.418220 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:46.418226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:46.418325 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:46.418634 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:46.419994 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.421455 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.423442 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.424337 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.424492 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:46.429386 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:46.429394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:46.429399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:46.429445 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:46.432851 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:46.433823 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:46.433828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:46.433832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:46.434006 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:46.434502 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:46.434507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:46.434510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:46.434968 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:46.435947 (+ 979us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:46.436106 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 00:34:46.436344 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15295,"thread_start_us":15767,"threads_started":1}
I20260812 00:34:46.437717 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:46.402498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42903 (local address 127.7.244.66:58047)
0812 00:34:46.403087 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:46.403111 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:46.410530 (+ 7419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:46.418692 (+ 8162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:46.418697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:46.419394 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:46.419790 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:46.419803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.421636 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.421643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:46.423221 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:46.423234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.424582 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.424589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:46.424796 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:46.425497 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:46.425522 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:46.429088 (+ 3566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:46.433012 (+ 3924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:46.433017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:46.433020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:46.433605 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:46.434143 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:46.434149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:46.434153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:46.434345 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:46.435118 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:46.435123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:46.435288 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 00:34:46.435418 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 00:34:46.436188 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":217,"thread_start_us":302,"threads_started":1}
I20260812 00:34:46.437728 22958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:46.400630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:52405)
0812 00:34:46.401790 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:46.401866 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:46.405054 (+ 3188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:46.406201 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:46.406216 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:46.407300 (+ 1084us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:46.408383 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:46.408416 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.415485 (+ 7069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.415491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:46.417618 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:46.417644 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.423231 (+ 5587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.423237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:46.423412 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:46.424052 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:46.424079 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:46.426701 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:46.431036 (+ 4335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:46.431040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:46.431043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:46.431826 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:46.432283 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:46.432287 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:46.432290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:46.432524 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:46.433355 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:46.433365 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:46.433968 (+ 603us) client_negotiation.cc:769] Sending connection context
0812 00:34:46.435431 (+ 1463us) client_negotiation.cc:240] Negotiation successful
0812 00:34:46.436053 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"mutex_wait_us":207,"spinlock_wait_cycles":12800,"thread_start_us":417,"threads_started":1}
I20260812 00:34:46.439064 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:46.400630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.7.244.66:41207)
0812 00:34:46.401790 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:46.401872 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:46.404695 (+ 2823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:46.409252 (+ 4557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:46.409258 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:46.409821 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:46.410143 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:46.410157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.411944 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.411950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:46.418559 (+ 6609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:46.418579 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.419797 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.419802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:46.420131 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:46.420978 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:46.421438 (+ 460us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:46.426235 (+ 4797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:46.430687 (+ 4452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:46.430698 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:46.430721 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:46.431705 (+ 984us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:46.432226 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:46.432231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:46.432234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:46.432524 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:46.433355 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:46.433365 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:46.433968 (+ 603us) client_negotiation.cc:769] Sending connection context
0812 00:34:46.435397 (+ 1429us) client_negotiation.cc:240] Negotiation successful
0812 00:34:46.438638 (+ 3241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":2694,"spinlock_wait_cycles":105728,"thread_start_us":452,"threads_started":1}
I20260812 00:34:46.440037 22952 heartbeater.cc:347] Connected to a master server at 127.7.244.125:40257
I20260812 00:34:46.440104 22950 heartbeater.cc:347] Connected to a master server at 127.7.244.124:42903
I20260812 00:34:46.440471 22950 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:46.440430 22952 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:46.440126 22951 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33751
I20260812 00:34:46.440644 22951 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:46.441573 22952 heartbeater.cc:511] Master 127.7.244.125:40257 requested a full tablet report, sending...
I20260812 00:34:46.442246 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:46.405055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53268 (local address 127.7.244.67:44261)
0812 00:34:46.418592 (+ 13537us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:46.418601 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:46.418635 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:34:46.418701 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:46.418710 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:46.419020 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:46.421138 (+ 2118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:46.421484 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.422564 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.424280 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:46.425192 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:46.426158 (+ 966us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:46.433155 (+ 6997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:46.433176 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:46.433200 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:46.433250 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:46.437807 (+ 4557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:46.438672 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:46.438682 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:46.438686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:46.438835 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:46.439624 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:46.439630 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:46.439633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:46.440119 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:46.440510 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:46.441538 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0812 00:34:46.441949 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13363,"thread_start_us":13200,"threads_started":1}
I20260812 00:34:46.442553 22951 heartbeater.cc:511] Master 127.7.244.126:33751 requested a full tablet report, sending...
I20260812 00:34:46.443058 22950 heartbeater.cc:511] Master 127.7.244.124:42903 requested a full tablet report, sending...
I20260812 00:34:46.443495 21935 ts_manager.cc:194] Registered new tserver with Master: 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217)
I20260812 00:34:46.443882 21657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:46.444140 20799 ts_manager.cc:194] Re-registered known tserver with Master: 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217)
I20260812 00:34:46.444298 22302 ts_manager.cc:194] Registered new tserver with Master: 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217)
I20260812 00:34:46.445335 20799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:52405
I20260812 00:34:46.529244 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21429
I20260812 00:34:46.561221 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:44261
--local_ip_for_outbound_sockets=127.7.244.67
--tserver_master_addrs=127.7.244.126:33751
--webserver_port=42997
--webserver_interface=127.7.244.67
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab
--principal=oryx/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257,127.7.244.124:42903 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:34:46.771076 22668 heartbeater.cc:503] Master 127.7.244.126:33751 was elected leader, sending a full tablet report...
W20260812 00:34:46.919132 22971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:46.919451 22971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:46.919574 22971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:46.927330 22971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:46.927470 22971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:46.927619 22971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:46.938757 22971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab
--principal=oryx/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:44261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=42997
--tserver_master_addrs=127.7.244.126:33751,127.7.244.125:40257,127.7.244.124:42903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:46.940408 22971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:46.941915 22971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:46.956600 22978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:46.956714 22983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:46.956511 22979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:46.957590 22971 server_base.cc:1034] running on GCE node
I20260812 00:34:46.958365 22971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:46.959445 22971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:46.960664 22971 hybrid_clock.cc:648] HybridClock initialized: now 1786494886960603 us; error 60 us; skew 500 ppm
I20260812 00:34:46.961024 22971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:46 dist-test-slave-gjw7 krb5kdc[20715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494886, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:46.964479 22971 init.cc:377] Logged in from keytab as oryx/127.7.244.67@KRBTEST.COM (short username oryx)
I20260812 00:34:46.966141 22971 webserver.cc:460] Webserver started at http://127.7.244.67:42997/ using document root <none> and password file <none>
I20260812 00:34:46.966934 22971 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:46.967082 22971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:46.973255 22971 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 00:34:46.976562 22996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:46.978253 22971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:34:46.978497 22971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "1e591840d9594679926354c22614d745"
format_stamp: "Formatted at 2026-08-12 00:34:43 on dist-test-slave-gjw7"
server_key: "d232a369e17d345960df2e01e4500208"
server_key_iv: "2032f7b15d15d5f6280ae30987aaea5c"
server_key_version: "encryptionkey@0"
I20260812 00:34:46.979610 22971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:46.996887 22971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:47.001674 22971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:47.002189 22971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:47.003441 22971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:34:47.005206 22971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:47.005373 22971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 00:34:47.005512 22971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:47.005599 22971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:47.042668 22971 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:44261
I20260812 00:34:47.042749 23193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:44261 every 8 connection(s)
I20260812 00:34:47.044267 22971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:47.048866 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 22971
Aug 12 00:34:47 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494886, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.125@KRBTEST.COM
Aug 12 00:34:47 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494886, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.124@KRBTEST.COM
I20260812 00:34:47.089167 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.047884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:49801 (local address 127.7.244.125:40257)
0812 00:34:47.066064 (+ 18180us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.066070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.066847 (+ 777us) server_negotiation.cc:407] Connection header received
0812 00:34:47.066917 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.066922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.067003 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.067101 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.068656 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.070175 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.073374 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.074022 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.074183 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.080519 (+ 6336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.080528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.080534 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.080587 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.084620 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.085933 (+ 1313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.085939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.085943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.086139 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.086758 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.086763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.086767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.087161 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.087295 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.088475 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.088744 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18046,"thread_start_us":17923,"threads_started":1}
I20260812 00:34:47.098608 23206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.048245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.7.244.67:49801)
0812 00:34:47.066568 (+ 18323us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.066597 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.066727 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.067317 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.067323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.068034 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:47.068411 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.068426 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.070468 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.070475 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.073053 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.073079 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.074599 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.074608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.074977 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.076036 (+ 1059us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:47.076075 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:47.080144 (+ 4069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:47.084969 (+ 4825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:47.084980 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:47.085001 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:47.085733 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:47.086293 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:47.086299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:47.086302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:47.086611 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:47.087412 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:47.087421 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:47.087966 (+ 545us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.089476 (+ 1510us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.090230 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18150,"mutex_wait_us":35,"spinlock_wait_cycles":20736,"thread_start_us":12142,"threads_started":1}
I20260812 00:34:47.100937 23202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.047494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:37609 (local address 127.7.244.124:42903)
0812 00:34:47.060160 (+ 12666us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.060168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.061474 (+ 1306us) server_negotiation.cc:407] Connection header received
0812 00:34:47.063789 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.063795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.063894 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.063996 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.066158 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.067261 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.077059 (+ 9798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.077770 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.077895 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.084838 (+ 6943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.084846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.084851 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.084904 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.089063 (+ 4159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.097049 (+ 7986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.097057 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.097061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.097338 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.098390 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.098395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.098398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.098824 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.098975 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.100178 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.100520 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12481,"thread_start_us":12696,"threads_started":1}
I20260812 00:34:47.101356 23200 heartbeater.cc:347] Connected to a master server at 127.7.244.125:40257
I20260812 00:34:47.101768 23200 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:47.102910 23200 heartbeater.cc:511] Master 127.7.244.125:40257 requested a full tablet report, sending...
I20260812 00:34:47.103444 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.048245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42903 (local address 127.7.244.67:37609)
0812 00:34:47.061047 (+ 12802us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.061110 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.063436 (+ 2326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.064338 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.064347 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.064979 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:47.065890 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.065923 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.073624 (+ 7701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.073628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.075098 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.075116 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.078239 (+ 3123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.078243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.078390 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.079359 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:47.079385 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:47.083384 (+ 3999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:47.095071 (+ 11687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:47.095078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:47.095083 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:47.095797 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:47.097577 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:47.097583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:47.097587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:47.097821 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:47.099098 (+ 1277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:47.099103 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:47.099273 (+ 170us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.101312 (+ 2039us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.101747 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12449,"thread_start_us":17833,"threads_started":1}
I20260812 00:34:47.105868 21935 ts_manager.cc:194] Registered new tserver with Master: 1e591840d9594679926354c22614d745 (127.7.244.67:44261)
Aug 12 00:34:47 dist-test-slave-gjw7 krb5kdc[20715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494886, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:47.113528 23194 heartbeater.cc:347] Connected to a master server at 127.7.244.124:42903
I20260812 00:34:47.113693 23194 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:47.114199 23194 heartbeater.cc:511] Master 127.7.244.124:42903 requested a full tablet report, sending...
I20260812 00:34:47.115712 22302 ts_manager.cc:194] Registered new tserver with Master: 1e591840d9594679926354c22614d745 (127.7.244.67:44261)
I20260812 00:34:47.127642 23206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.066896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:43401)
0812 00:34:47.101111 (+ 34215us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.101132 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.101250 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.102322 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.102327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.102946 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:47.103158 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.103171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.106400 (+ 3229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.106407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.107984 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.108002 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.109427 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.109434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.109643 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.110381 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:47.110405 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:47.117056 (+ 6651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:47.121972 (+ 4916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:47.121978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:47.121982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:47.122550 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:47.125068 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:47.125072 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:47.125075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:47.125247 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:47.126460 (+ 1213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:47.126466 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:47.126680 (+ 214us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.126850 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.127223 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34094,"thread_start_us":44106,"threads_started":1}
I20260812 00:34:47.127931 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.067655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:43401 (local address 127.7.244.126:33751)
0812 00:34:47.084541 (+ 16886us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.084548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.101213 (+ 16665us) server_negotiation.cc:407] Connection header received
0812 00:34:47.101894 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.101902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.102005 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.102291 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.103700 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.106241 (+ 2541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.108448 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.109229 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.109421 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.117285 (+ 7864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.117336 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.117340 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.117390 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.121749 (+ 4359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.122691 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.122697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.122701 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.122877 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.125406 (+ 2529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.125670 (+ 264us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.125673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.126120 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.126275 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.127047 (+ 772us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.127528 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16727,"thread_start_us":16895,"threads_started":1}
I20260812 00:34:47.128829 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.092325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.7.244.125:40257)
0812 00:34:47.093255 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.093262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.108184 (+ 14922us) server_negotiation.cc:407] Connection header received
0812 00:34:47.108426 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.108433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.108550 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.108652 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.111391 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.112553 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.115857 (+ 3304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.116597 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.116718 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.121630 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.121638 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.121643 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.121694 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.124848 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.125761 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.125767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.125770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.125943 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.126649 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.126655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.126658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.127042 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.127301 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.128219 (+ 918us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.128495 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743}
I20260812 00:34:47.129106 23196 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33751
I20260812 00:34:47.129248 23196 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:47.129845 23196 heartbeater.cc:511] Master 127.7.244.126:33751 requested a full tablet report, sending...
I20260812 00:34:47.130998 20799 ts_manager.cc:194] Re-registered known tserver with Master: 1e591840d9594679926354c22614d745 (127.7.244.67:44261)
I20260812 00:34:47.131906 20799 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:43401
I20260812 00:34:47.135670 23202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.099661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43296 (local address 127.7.244.124:42903)
0812 00:34:47.101153 (+ 1492us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.101158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.117944 (+ 16786us) server_negotiation.cc:407] Connection header received
0812 00:34:47.118060 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.118066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.118172 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.118282 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.121018 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.122118 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.124037 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.124721 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.124866 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.128075 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.128084 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.128088 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.128133 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.131975 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.132797 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.132804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.132807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.132984 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.133461 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.133466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.133469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.133865 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.134096 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.134687 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.134950 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1403,"thread_start_us":8327,"threads_started":1}
I20260812 00:34:47.137099 23221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.086107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.7.244.126:33751)
0812 00:34:47.110556 (+ 24449us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.110563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.110582 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:34:47.110638 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.110644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.110721 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.114798 (+ 4077us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.115059 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.116204 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.120357 (+ 4153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.121119 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.121261 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.125643 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.125650 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.125656 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.125690 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.132779 (+ 7089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.133659 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.133664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.133667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.133841 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.134312 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.134317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.134320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.135373 (+ 1053us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.135636 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.136381 (+ 745us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.136652 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24313,"thread_start_us":24150,"threads_started":1}
I20260812 00:34:47.141893 20799 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46526:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:47.149629 20799 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:47.185586 23247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.172654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:44261 (local address 127.0.0.1:53272)
0812 00:34:47.173528 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.173551 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.173693 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.176388 (+ 2695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.176392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.176417 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:47.176834 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.176847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.179828 (+ 2981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.179834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.182506 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.182522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.184442 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.184448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.184681 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.184722 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.184823 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.184975 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":25,"thread_start_us":837,"threads_started":1}
I20260812 00:34:47.185577 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.173279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.7.244.67:44261)
0812 00:34:47.175556 (+ 2277us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.175566 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.175608 (+ 42us) server_negotiation.cc:407] Connection header received
0812 00:34:47.175749 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.175758 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.176106 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.176423 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:47.177064 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.179265 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.182740 (+ 3475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.183978 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.184119 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.184474 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.184890 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.185103 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2024,"thread_start_us":1798,"threads_started":1}
I20260812 00:34:47.187634 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.173814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.7.244.65:44493)
0812 00:34:47.175660 (+ 1846us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.175669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.177653 (+ 1984us) server_negotiation.cc:407] Connection header received
0812 00:34:47.177797 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.177806 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.178175 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.178397 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:47.179522 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.181530 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.184283 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.186441 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.186570 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.186826 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.187055 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.187239 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":696,"threads_started":1}
I20260812 00:34:47.187702 23249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.173136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44493 (local address 127.0.0.1:46090)
0812 00:34:47.177314 (+ 4178us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.177335 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.177494 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.178756 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.178762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.178794 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:47.179214 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.179226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.181740 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.181747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.184094 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.184110 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.186653 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.186660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.186872 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.186906 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.186988 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.187126 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3956,"mutex_wait_us":32,"thread_start_us":3781,"threads_started":1}
I20260812 00:34:47.190011 23076 tablet_service.cc:1467] Processing CreateTablet for tablet e2cfe6436ff74133a1edaf4720d6907b (DEFAULT_TABLE table=test-table [id=26b3bce4bcd24663b95b79bf3484d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:47.191766 23076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2cfe6436ff74133a1edaf4720d6907b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:47.192251 22527 tablet_service.cc:1467] Processing CreateTablet for tablet e2cfe6436ff74133a1edaf4720d6907b (DEFAULT_TABLE table=test-table [id=26b3bce4bcd24663b95b79bf3484d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:47.194068 22527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2cfe6436ff74133a1edaf4720d6907b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:47.199034 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.177382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:45217 (local address 127.0.0.1:46942)
0812 00:34:47.184776 (+ 7394us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.186676 (+ 1900us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.186970 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.188095 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.188100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.188127 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:47.188469 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.188481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.194289 (+ 5808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.194295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.195936 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.195963 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.198159 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.198166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.198351 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.198380 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.198471 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.198622 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7256,"thread_start_us":5115,"threads_started":1}
I20260812 00:34:47.199083 23256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.177706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46942 (local address 127.7.244.66:45217)
0812 00:34:47.181757 (+ 4051us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.181766 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.186835 (+ 5069us) server_negotiation.cc:407] Connection header received
0812 00:34:47.187194 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.187203 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.187487 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.187782 (+ 295us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:47.188690 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.190395 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.196559 (+ 6164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.197922 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.198072 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.198293 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.198537 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.198701 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3851,"thread_start_us":4158,"threads_started":1}
I20260812 00:34:47.203140 22803 tablet_service.cc:1467] Processing CreateTablet for tablet e2cfe6436ff74133a1edaf4720d6907b (DEFAULT_TABLE table=test-table [id=26b3bce4bcd24663b95b79bf3484d1b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:47.204703 22803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2cfe6436ff74133a1edaf4720d6907b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:47.211663 23257 tablet_bootstrap.cc:492] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Bootstrap starting.
I20260812 00:34:47.214124 23259 tablet_bootstrap.cc:492] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Bootstrap starting.
I20260812 00:34:47.218026 23259 tablet_bootstrap.cc:654] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:47.218101 23257 tablet_bootstrap.cc:654] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:47.219455 23259 log.cc:824] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:47.219856 23257 log.cc:824] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:47.222879 23265 tablet_bootstrap.cc:492] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Bootstrap starting.
I20260812 00:34:47.223587 23259 tablet_bootstrap.cc:492] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: No bootstrap required, opened a new log
I20260812 00:34:47.223958 23259 ts_tablet_manager.cc:1397] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260812 00:34:47.224253 23257 tablet_bootstrap.cc:492] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: No bootstrap required, opened a new log
I20260812 00:34:47.224542 23257 ts_tablet_manager.cc:1397] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260812 00:34:47.227711 23265 tablet_bootstrap.cc:654] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:47.229354 23265 log.cc:824] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:47.233605 23265 tablet_bootstrap.cc:492] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: No bootstrap required, opened a new log
I20260812 00:34:47.234004 23265 ts_tablet_manager.cc:1397] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260812 00:34:47.234311 23259 raft_consensus.cc:348] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.234731 23259 raft_consensus.cc:374] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:47.234829 23259 raft_consensus.cc:729] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e591840d9594679926354c22614d745, State: Initialized, Role: FOLLOWER
I20260812 00:34:47.235545 23259 consensus_queue.cc:260] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.236694 23257 raft_consensus.cc:348] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.237200 23257 raft_consensus.cc:374] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:47.237325 23257 raft_consensus.cc:729] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e640fd14b3b1411fa515c1cdbd2d113d, State: Initialized, Role: FOLLOWER
I20260812 00:34:47.237625 23259 ts_tablet_manager.cc:1428] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Time spent starting tablet: real 0.014s user 0.012s sys 0.004s
I20260812 00:34:47.237948 23196 heartbeater.cc:503] Master 127.7.244.126:33751 was elected leader, sending a full tablet report...
I20260812 00:34:47.239341 23257 consensus_queue.cc:260] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.245878 23257 ts_tablet_manager.cc:1428] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Time spent starting tablet: real 0.021s user 0.014s sys 0.005s
I20260812 00:34:47.248974 23265 raft_consensus.cc:348] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.249814 23265 raft_consensus.cc:374] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:47.249951 23265 raft_consensus.cc:729] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e9e1797474436ab2a79319877c53dd, State: Initialized, Role: FOLLOWER
I20260812 00:34:47.250763 23265 consensus_queue.cc:260] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.252169 22951 heartbeater.cc:503] Master 127.7.244.126:33751 was elected leader, sending a full tablet report...
I20260812 00:34:47.252966 23265 ts_tablet_manager.cc:1428] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Time spent starting tablet: real 0.019s user 0.012s sys 0.004s
W20260812 00:34:47.297783 23201 tablet.cc:2369] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:34:47.399963 22953 tablet.cc:2369] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:47.454077 23268 raft_consensus.cc:482] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:47.454422 23268 raft_consensus.cc:504] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.456398 23268 leader_election.cc:288] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e591840d9594679926354c22614d745 (127.7.244.67:44261), 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217)
I20260812 00:34:47.461627 23269 raft_consensus.cc:482] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:47.462082 23269 raft_consensus.cc:504] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.464941 23269 leader_election.cc:288] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e591840d9594679926354c22614d745 (127.7.244.67:44261), e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65:44493)
I20260812 00:34:47.468132 23256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.457039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:60829 (local address 127.7.244.66:45217)
0812 00:34:47.457248 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.457254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.458323 (+ 1069us) server_negotiation.cc:407] Connection header received
0812 00:34:47.458500 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.458504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.458569 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.458646 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:47.459623 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.460489 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.464588 (+ 4099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.466515 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.466661 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.466762 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.467534 (+ 772us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.467715 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 00:34:47.468724 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.457008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:42479 (local address 127.7.244.67:44261)
0812 00:34:47.457394 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.457400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.458269 (+ 869us) server_negotiation.cc:407] Connection header received
0812 00:34:47.458505 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.458510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.458575 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.458655 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:47.459656 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.460637 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.464537 (+ 3900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.465985 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.466180 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.466279 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.468140 (+ 1861us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.468315 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260812 00:34:47.469585 23272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.456929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:45217 (local address 127.7.244.65:60829)
0812 00:34:47.458186 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.458201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.458351 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.458688 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.458692 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.458742 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:47.459388 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.459405 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.460642 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.460647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.464334 (+ 3687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.464347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.467062 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.467069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.467278 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.467322 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.468462 (+ 1140us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.468638 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":194,"spinlock_wait_cycles":2304,"thread_start_us":878,"threads_started":1}
I20260812 00:34:47.469980 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.456618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:44261 (local address 127.7.244.65:42479)
0812 00:34:47.458138 (+ 1520us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.458156 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.458316 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.458692 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.458696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.458747 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:47.459501 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.459514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.460892 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.460897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.464221 (+ 3324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.464251 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.467548 (+ 3297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.467557 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.467899 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.467940 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.469030 (+ 1090us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.469209 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1390,"mutex_wait_us":110,"spinlock_wait_cycles":31232,"thread_start_us":871,"threads_started":1}
I20260812 00:34:47.471320 22851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2cfe6436ff74133a1edaf4720d6907b" candidate_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e9e1797474436ab2a79319877c53dd" is_pre_election: true
I20260812 00:34:47.472095 22851 raft_consensus.cc:2454] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e640fd14b3b1411fa515c1cdbd2d113d in term 0.
I20260812 00:34:47.473225 22435 leader_election.cc:302] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94e9e1797474436ab2a79319877c53dd, e640fd14b3b1411fa515c1cdbd2d113d; no voters:
I20260812 00:34:47.473158 23119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2cfe6436ff74133a1edaf4720d6907b" candidate_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e591840d9594679926354c22614d745" is_pre_election: true
I20260812 00:34:47.473846 23268 raft_consensus.cc:2781] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:47.473994 23268 raft_consensus.cc:482] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:47.473912 23119 raft_consensus.cc:2454] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e640fd14b3b1411fa515c1cdbd2d113d in term 0.
I20260812 00:34:47.474083 23268 raft_consensus.cc:3037] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:47.476624 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.464327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:44261 (local address 127.7.244.66:53969)
0812 00:34:47.465810 (+ 1483us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.465835 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.466018 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.467762 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.467769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.467805 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:47.468380 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.468392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.471228 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.471235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.473703 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.473723 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.475938 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.475946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.476092 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.476128 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.476212 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.476338 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"threads_started":1}
I20260812 00:34:47.477665 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.466435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:53969 (local address 127.7.244.67:44261)
0812 00:34:47.467382 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.467388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.467423 (+ 35us) server_negotiation.cc:407] Connection header received
0812 00:34:47.467496 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.467502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.467584 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.468539 (+ 955us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:47.468977 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.471021 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.473934 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.475588 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.476908 (+ 1320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.476994 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.477049 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.477237 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":605,"threads_started":1}
I20260812 00:34:47.477581 23268 raft_consensus.cc:504] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.478283 23119 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2cfe6436ff74133a1edaf4720d6907b" candidate_uuid: "94e9e1797474436ab2a79319877c53dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e591840d9594679926354c22614d745" is_pre_election: true
I20260812 00:34:47.478606 23119 raft_consensus.cc:2454] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94e9e1797474436ab2a79319877c53dd in term 0.
I20260812 00:34:47.478860 23113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2cfe6436ff74133a1edaf4720d6907b" candidate_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e591840d9594679926354c22614d745"
I20260812 00:34:47.479084 23113 raft_consensus.cc:3037] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:47.479904 22851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2cfe6436ff74133a1edaf4720d6907b" candidate_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e9e1797474436ab2a79319877c53dd"
I20260812 00:34:47.480201 22851 raft_consensus.cc:3037] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:47.480260 23268 leader_election.cc:288] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [CANDIDATE]: Term 1 election: Requested vote from peers 1e591840d9594679926354c22614d745 (127.7.244.67:44261), 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217)
I20260812 00:34:47.482071 23113 raft_consensus.cc:2454] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e640fd14b3b1411fa515c1cdbd2d113d in term 1.
I20260812 00:34:47.482666 22432 leader_election.cc:302] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e591840d9594679926354c22614d745, e640fd14b3b1411fa515c1cdbd2d113d; no voters:
I20260812 00:34:47.482645 22720 leader_election.cc:302] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e591840d9594679926354c22614d745, 94e9e1797474436ab2a79319877c53dd; no voters:
I20260812 00:34:47.483098 23268 raft_consensus.cc:2781] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 1 FOLLOWER]: Leader election won for term 1
W20260812 00:34:47.483217 22673 tablet.cc:2369] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:47.484854 22851 raft_consensus.cc:2454] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e640fd14b3b1411fa515c1cdbd2d113d in term 1.
I20260812 00:34:47.485139 23269 raft_consensus.cc:2741] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260812 00:34:47.485356 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.464569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:55315 (local address 127.7.244.65:44493)
0812 00:34:47.464775 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.464781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.476683 (+ 11902us) server_negotiation.cc:407] Connection header received
0812 00:34:47.476884 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.476890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.476979 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.477079 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:47.478102 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.479222 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.482511 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.483820 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.483960 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.484014 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.484686 (+ 672us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.484850 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 00:34:47.485968 23268 raft_consensus.cc:686] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 1 LEADER]: Becoming Leader. State: Replica: e640fd14b3b1411fa515c1cdbd2d113d, State: Running, Role: LEADER
I20260812 00:34:47.486006 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.464327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44493 (local address 127.7.244.66:55315)
0812 00:34:47.476472 (+ 12145us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:47.476500 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:47.476717 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:47.477267 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:47.477272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:47.477361 (+ 89us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:47.477880 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.477896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.479417 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.479422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.482251 (+ 2829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:47.482277 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.484158 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.484175 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:47.484398 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.484437 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 00:34:47.485044 (+ 607us) client_negotiation.cc:240] Negotiation successful
0812 00:34:47.485233 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11990,"mutex_wait_us":374,"thread_start_us":601,"threads_started":1}
I20260812 00:34:47.487010 23268 consensus_queue.cc:237] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } }
I20260812 00:34:47.487891 22575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2cfe6436ff74133a1edaf4720d6907b" candidate_uuid: "94e9e1797474436ab2a79319877c53dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" is_pre_election: true
I20260812 00:34:47.493269 20797 catalog_manager.cc:5495] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d reported cstate change: term changed from 0 to 1, leader changed from <none> to e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65). New cstate: current_term: 1 leader_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } health_report { overall_health: UNKNOWN } } }
I20260812 00:34:47.565585 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.559360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46098 (local address 127.7.244.65:44493)
0812 00:34:47.559551 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.559556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.559823 (+ 267us) server_negotiation.cc:407] Connection header received
0812 00:34:47.560005 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.560010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.560099 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.560255 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:47.560644 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.561674 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.563147 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.563737 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.563852 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.564431 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:47.564992 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:47.565118 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.565179 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.565258 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 00:34:47.575740 23113 raft_consensus.cc:1261] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 1 FOLLOWER]: Refusing update from remote peer e640fd14b3b1411fa515c1cdbd2d113d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:47.575809 22851 raft_consensus.cc:1261] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Refusing update from remote peer e640fd14b3b1411fa515c1cdbd2d113d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:47.577082 23278 consensus_queue.cc:1035] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:47.577585 23268 consensus_queue.cc:1035] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [LEADER]: Connected to new peer: Peer: permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:47.596062 23283 mvcc.cc:205] Tried to move back new op lower bound from 7317483059497037824 to 7317483059151298560. Current Snapshot: MvccSnapshot[applied={T|T < 7317483059497037824}]
I20260812 00:34:47.597499 23288 mvcc.cc:205] Tried to move back new op lower bound from 7317483059497037824 to 7317483059151298560. Current Snapshot: MvccSnapshot[applied={T|T < 7317483059497037824}]
I20260812 00:34:47.617273 20797 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:47.617703 20797 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46526:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:47.625166 20797 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57280ef1740948289e04fb0a6dd73669: Sending DeleteTablet for 3 replicas of tablet e2cfe6436ff74133a1edaf4720d6907b
I20260812 00:34:47.626456 23076 tablet_service.cc:1514] Processing DeleteTablet for tablet e2cfe6436ff74133a1edaf4720d6907b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:47 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:53272
I20260812 00:34:47.627465 23311 tablet_replica.cc:330] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: stopping tablet replica
I20260812 00:34:47.628185 23311 raft_consensus.cc:2229] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:47.628175 22527 tablet_service.cc:1514] Processing DeleteTablet for tablet e2cfe6436ff74133a1edaf4720d6907b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:47 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:46090
I20260812 00:34:47.628901 22803 tablet_service.cc:1514] Processing DeleteTablet for tablet e2cfe6436ff74133a1edaf4720d6907b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:47 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:46942
I20260812 00:34:47.629021 23311 raft_consensus.cc:2258] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:47.629751 23313 tablet_replica.cc:330] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: stopping tablet replica
I20260812 00:34:47.630311 23313 raft_consensus.cc:2229] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:47.631238 23313 raft_consensus.cc:2258] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:47.632790 23311 ts_tablet_manager.cc:1905] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:47.633651 23313 ts_tablet_manager.cc:1905] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:47.639125 23318 tablet_replica.cc:330] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: stopping tablet replica
I20260812 00:34:47.639693 23318 raft_consensus.cc:2229] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:47.640504 23318 raft_consensus.cc:2258] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:47.641438 23311 ts_tablet_manager.cc:1918] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:47.641768 23311 log.cc:1196] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/e2cfe6436ff74133a1edaf4720d6907b
I20260812 00:34:47.642573 23311 ts_tablet_manager.cc:1939] T e2cfe6436ff74133a1edaf4720d6907b P 1e591840d9594679926354c22614d745: Deleting consensus metadata
I20260812 00:34:47.644605 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20737
I20260812 00:34:47.645471 20768 catalog_manager.cc:4841] TS 1e591840d9594679926354c22614d745 (127.7.244.67:44261): tablet e2cfe6436ff74133a1edaf4720d6907b (table test-table [id=26b3bce4bcd24663b95b79bf3484d1b6]) successfully deleted
I20260812 00:34:47.646395 20852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:47.646991 23313 ts_tablet_manager.cc:1918] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:47.647352 23313 log.cc:1196] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/e2cfe6436ff74133a1edaf4720d6907b
I20260812 00:34:47.650177 23318 ts_tablet_manager.cc:1905] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:47.651626 23313 ts_tablet_manager.cc:1939] T e2cfe6436ff74133a1edaf4720d6907b P e640fd14b3b1411fa515c1cdbd2d113d: Deleting consensus metadata
I20260812 00:34:47.662516 23318 ts_tablet_manager.cc:1918] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:47.662750 23318 log.cc:1196] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/e2cfe6436ff74133a1edaf4720d6907b
I20260812 00:34:47.663395 23318 ts_tablet_manager.cc:1939] T e2cfe6436ff74133a1edaf4720d6907b P 94e9e1797474436ab2a79319877c53dd: Deleting consensus metadata
I20260812 00:34:47.669716 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.630938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34334 (local address 127.7.244.125:40257)
0812 00:34:47.645902 (+ 14964us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.645908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.645931 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:34:47.645996 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.646000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.646070 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.646201 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.648544 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.649599 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.652201 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.652926 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.653063 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.661333 (+ 8270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.661342 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.661347 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.661405 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.665917 (+ 4512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.666991 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.666997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.667002 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.667182 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.667686 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.667689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.667692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.668157 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.668624 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.668840 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.669180 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14756,"thread_start_us":14784,"threads_started":1}
I20260812 00:34:47.689044 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.641504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.7.244.124:42903)
0812 00:34:47.645094 (+ 3590us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.645101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.669748 (+ 24647us) server_negotiation.cc:407] Connection header received
0812 00:34:47.670040 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.670047 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.670140 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.670225 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.672347 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.673861 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.676175 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.676980 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.677125 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.681148 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.681155 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.681160 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.681208 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.685687 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.686645 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.686651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.686655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.686831 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.687325 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.687331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.687341 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.687753 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.688033 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.688322 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.688585 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3449,"thread_start_us":3849,"threads_started":1}
I20260812 00:34:47.797029 20773 catalog_manager.cc:4841] TS 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217): tablet e2cfe6436ff74133a1edaf4720d6907b (table test-table [id=26b3bce4bcd24663b95b79bf3484d1b6]) successfully deleted
I20260812 00:34:47.797526 20773 catalog_manager.cc:4841] TS e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65:44493): tablet e2cfe6436ff74133a1edaf4720d6907b (table test-table [id=26b3bce4bcd24663b95b79bf3484d1b6]) successfully deleted
I20260812 00:34:47.798139 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20737
I20260812 00:34:47.851497 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.832318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.7.244.125:40257)
0812 00:34:47.832577 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.832583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.834067 (+ 1484us) server_negotiation.cc:407] Connection header received
0812 00:34:47.834430 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.834436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.834535 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.834613 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.837539 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.838277 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.840211 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.840768 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.840858 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.845011 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.845017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.845021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.845067 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.848056 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.848919 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.848924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.848926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.849106 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.849617 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.849623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.849626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.849949 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.850213 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.850873 (+ 660us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.851133 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 00:34:47.853094 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:47.833435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.7.244.124:42903)
0812 00:34:47.833625 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:47.833631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:47.834537 (+ 906us) server_negotiation.cc:407] Connection header received
0812 00:34:47.834851 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:47.834856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:47.834947 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:47.835059 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:47.837474 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.838642 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.840461 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:47.841214 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:47.841392 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:47.845050 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:47.845060 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:47.845065 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:47.845117 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:47.848780 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.850022 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.850029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.850032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.850267 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:47.850714 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:47.850719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:47.850722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:47.851107 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:47.851355 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:47.852479 (+ 1124us) server_negotiation.cc:299] Negotiation successful
0812 00:34:47.852742 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:34:48.605322 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:48.604047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:54355)
0812 00:34:48.604801 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:48.604967 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":246,"threads_started":1}
W20260812 00:34:48.605877 23196 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:33751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:48.605938 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:48.604951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:49219)
0812 00:34:48.605575 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:48.605714 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":364,"threads_started":1}
W20260812 00:34:48.606655 22951 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:33751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:48.610010 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:48.609143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:40065)
0812 00:34:48.609653 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:48.609801 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":305,"threads_started":1}
W20260812 00:34:48.610715 22668 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:33751 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:49.126096 23345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 57280ef1740948289e04fb0a6dd73669)
I20260812 00:34:49.126171 23347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 57280ef1740948289e04fb0a6dd73669)
I20260812 00:34:49.126338 23345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } }
I20260812 00:34:49.126474 23347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } }
I20260812 00:34:49.127194 23345 leader_election.cc:288] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751), 5ecdfb3b4e0f4487873207bd5c590384 (127.7.244.125:40257)
I20260812 00:34:49.134565 23347 leader_election.cc:288] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751), 901445618aa9460191789ee586194f48 (127.7.244.124:42903)
I20260812 00:34:49.138691 23348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.127417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:46990)
0812 00:34:49.137842 (+ 10425us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.138081 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10260,"threads_started":1}
I20260812 00:34:49.139412 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.128203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:46998)
0812 00:34:49.138850 (+ 10647us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.139087 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10518,"thread_start_us":10383,"threads_started":1}
W20260812 00:34:49.139696 22272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
W20260812 00:34:49.141043 21910 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:49.148481 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.127604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34348 (local address 127.7.244.125:40257)
0812 00:34:49.137840 (+ 10236us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:49.137848 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:49.138585 (+ 737us) server_negotiation.cc:407] Connection header received
0812 00:34:49.138936 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:49.138942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:49.139093 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:49.139185 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:49.140213 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:49.141407 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:49.145057 (+ 3650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:49.146306 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:49.146406 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:49.146516 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:49.147860 (+ 1344us) server_negotiation.cc:299] Negotiation successful
0812 00:34:49.148034 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10069,"thread_start_us":10178,"threads_started":1}
I20260812 00:34:49.148964 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.127505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:40257 (local address 127.0.0.1:34348)
0812 00:34:49.138437 (+ 10932us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.138460 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:49.138734 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:49.139461 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:49.139466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:49.139500 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:49.139961 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:49.139980 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:49.141597 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:49.141602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:49.144851 (+ 3249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:49.144869 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:49.146648 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:49.146657 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:49.146849 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:49.147690 (+ 841us) client_negotiation.cc:769] Sending connection context
0812 00:34:49.147810 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 00:34:49.147980 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10795,"mutex_wait_us":773,"thread_start_us":10637,"threads_started":1}
I20260812 00:34:49.149071 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.140191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42903 (local address 127.0.0.1:43334)
0812 00:34:49.140415 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.140441 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:49.140602 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:49.141005 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:49.141011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:49.141043 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:49.141420 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:49.141430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:49.143347 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:49.143353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:49.145394 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:49.145421 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:49.148249 (+ 2828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:49.148256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:49.148495 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:49.148531 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 00:34:49.148657 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:34:49.148812 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 00:34:49.149768 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.139261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.7.244.124:42903)
0812 00:34:49.140224 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:49.140230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:49.140549 (+ 319us) server_negotiation.cc:407] Connection header received
0812 00:34:49.140754 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:49.140759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:49.140832 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:49.141547 (+ 715us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:49.141864 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:49.143177 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:49.146561 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:49.147870 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:49.147990 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:49.148098 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:49.149248 (+ 1150us) server_negotiation.cc:299] Negotiation successful
0812 00:34:49.149427 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":838,"thread_start_us":685,"threads_started":1}
I20260812 00:34:49.149693 21953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "901445618aa9460191789ee586194f48" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ecdfb3b4e0f4487873207bd5c590384" is_pre_election: true
I20260812 00:34:49.150272 21953 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901445618aa9460191789ee586194f48 in term 1.
I20260812 00:34:49.150681 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.149532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:47026)
0812 00:34:49.150397 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.150521 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":750}
I20260812 00:34:49.150640 22318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ecdfb3b4e0f4487873207bd5c590384" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "901445618aa9460191789ee586194f48" is_pre_election: true
W20260812 00:34:49.150872 22272 leader_election.cc:334] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751): Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:49.151041 22318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ecdfb3b4e0f4487873207bd5c590384 in term 1.
I20260812 00:34:49.151201 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.147803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:47022)
0812 00:34:49.150717 (+ 2914us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.150921 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1772,"thread_start_us":1857,"threads_started":1}
I20260812 00:34:49.151163 22270 leader_election.cc:302] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ecdfb3b4e0f4487873207bd5c590384, 901445618aa9460191789ee586194f48; no voters: 57280ef1740948289e04fb0a6dd73669
W20260812 00:34:49.151427 21910 leader_election.cc:334] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751): Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:49.151526 23345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 00:34:49.151635 23345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 57280ef1740948289e04fb0a6dd73669)
I20260812 00:34:49.151731 23345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:34:49.151845 21910 leader_election.cc:302] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ecdfb3b4e0f4487873207bd5c590384, 901445618aa9460191789ee586194f48; no voters: 57280ef1740948289e04fb0a6dd73669
I20260812 00:34:49.152225 23347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 00:34:49.152351 23347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 57280ef1740948289e04fb0a6dd73669)
I20260812 00:34:49.152453 23347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:34:49.154168 23345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } }
I20260812 00:34:49.154659 23347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } }
I20260812 00:34:49.154855 23345 leader_election.cc:288] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [CANDIDATE]: Term 2 election: Requested vote from peers 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751), 5ecdfb3b4e0f4487873207bd5c590384 (127.7.244.125:40257)
I20260812 00:34:49.155351 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.154971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:47034)
0812 00:34:49.155116 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.155204 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 00:34:49.155426 23347 leader_election.cc:288] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 2 election: Requested vote from peers 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751), 901445618aa9460191789ee586194f48 (127.7.244.124:42903)
I20260812 00:34:49.155546 21953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "901445618aa9460191789ee586194f48" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ecdfb3b4e0f4487873207bd5c590384"
I20260812 00:34:49.155859 21953 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 901445618aa9460191789ee586194f48 in current term 2: Already voted for candidate 5ecdfb3b4e0f4487873207bd5c590384 in this term.
I20260812 00:34:49.155962 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.155556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:47036)
0812 00:34:49.155692 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.155780 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 00:34:49.156342 22318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ecdfb3b4e0f4487873207bd5c590384" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "901445618aa9460191789ee586194f48"
I20260812 00:34:49.156641 22318 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ecdfb3b4e0f4487873207bd5c590384 in current term 2: Already voted for candidate 901445618aa9460191789ee586194f48 in this term.
I20260812 00:34:49.157141 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.156742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:47040)
0812 00:34:49.156877 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.156993 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 00:34:49.157397 22272 leader_election.cc:334] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751): Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:49.157553 22272 leader_election.cc:302] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 901445618aa9460191789ee586194f48; no voters: 57280ef1740948289e04fb0a6dd73669, 5ecdfb3b4e0f4487873207bd5c590384
I20260812 00:34:49.157804 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.157379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:47046)
0812 00:34:49.157556 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.157656 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 00:34:49.157851 23345 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260812 00:34:49.157956 21910 leader_election.cc:334] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751): Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:49.158079 21910 leader_election.cc:302] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ecdfb3b4e0f4487873207bd5c590384; no voters: 57280ef1740948289e04fb0a6dd73669, 901445618aa9460191789ee586194f48
I20260812 00:34:49.158347 23347 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260812 00:34:49.607384 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.606598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:36375)
0812 00:34:49.607089 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.607204 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
I20260812 00:34:49.608134 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.607354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:44783)
0812 00:34:49.607828 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.607931 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":255,"threads_started":1}
I20260812 00:34:49.612464 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:49.611382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:57277)
0812 00:34:49.612163 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:49.612251 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":510,"threads_started":1}
I20260812 00:34:50.609169 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.608251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:47977)
0812 00:34:50.608742 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.608870 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":246,"threads_started":1}
I20260812 00:34:50.609853 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.609015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:58107)
0812 00:34:50.609582 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.609679 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":295,"threads_started":1}
I20260812 00:34:50.614221 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.613245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:53381)
0812 00:34:50.613845 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.613985 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":372,"threads_started":1}
I20260812 00:34:50.900204 23374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:50.900465 23374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } }
I20260812 00:34:50.901225 23374 leader_election.cc:288] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751), 901445618aa9460191789ee586194f48 (127.7.244.124:42903)
I20260812 00:34:50.902518 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.901522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:38212)
0812 00:34:50.902104 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.902250 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":270,"threads_started":1}
I20260812 00:34:50.902490 22318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ecdfb3b4e0f4487873207bd5c590384" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "901445618aa9460191789ee586194f48" is_pre_election: true
I20260812 00:34:50.902806 22318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ecdfb3b4e0f4487873207bd5c590384 in term 2.
I20260812 00:34:50.903571 21910 leader_election.cc:302] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ecdfb3b4e0f4487873207bd5c590384, 901445618aa9460191789ee586194f48; no voters:
I20260812 00:34:50.903904 23374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 00:34:50.904027 23374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:50.904166 23374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 2 FOLLOWER]: Advancing to term 3
I20260812 00:34:50.904407 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.904023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:38238)
0812 00:34:50.904170 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.904253 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 00:34:50.904618 21910 leader_election.cc:334] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751): Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:50.906580 23374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } }
I20260812 00:34:50.907250 23374 leader_election.cc:288] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 3 election: Requested vote from peers 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751), 901445618aa9460191789ee586194f48 (127.7.244.124:42903)
I20260812 00:34:50.908088 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.907555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:38250)
0812 00:34:50.907839 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.907928 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 00:34:50.908255 22318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ecdfb3b4e0f4487873207bd5c590384" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "901445618aa9460191789ee586194f48"
I20260812 00:34:50.908582 22318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 2 FOLLOWER]: Advancing to term 3
I20260812 00:34:50.909197 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.908807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:38262)
0812 00:34:50.908988 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.909069 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260812 00:34:50.909410 21910 leader_election.cc:334] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751): Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
I20260812 00:34:50.911116 22318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ecdfb3b4e0f4487873207bd5c590384 in term 3.
I20260812 00:34:50.911653 21910 leader_election.cc:302] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ecdfb3b4e0f4487873207bd5c590384, 901445618aa9460191789ee586194f48; no voters: 57280ef1740948289e04fb0a6dd73669
I20260812 00:34:50.912019 23374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 00:34:50.912492 23374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [term 3 LEADER]: Becoming Leader. State: Replica: 5ecdfb3b4e0f4487873207bd5c590384, State: Running, Role: LEADER
I20260812 00:34:50.912886 23374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } }
I20260812 00:34:50.914902 23382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ecdfb3b4e0f4487873207bd5c590384. Latest consensus state: current_term: 3 leader_uuid: "5ecdfb3b4e0f4487873207bd5c590384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:50.915259 23382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:50.915946 23384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:50.919904 23384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:50.920773 23384 catalog_manager.cc:1259] Loaded cluster ID: a6ce5fba19344004b179acb4e647ef35
I20260812 00:34:50.920867 23384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:50.922132 23384 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:50.922976 23384 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Loaded TSK: 0
I20260812 00:34:50.923853 23384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:34:50.985531 21935 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:50.992724 21935 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:50.997195 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.996805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:38284)
0812 00:34:50.996948 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.997038 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 00:34:50.997705 22318 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [term 3 FOLLOWER]: Refusing update from remote peer 5ecdfb3b4e0f4487873207bd5c590384: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260812 00:34:50.998371 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:50.997965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:38294)
0812 00:34:50.998144 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:50.998244 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 00:34:50.998462 23374 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [LEADER]: Connected to new peer: Peer: permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 00:34:50.998795 21910 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 -> Peer 57280ef1740948289e04fb0a6dd73669 (127.7.244.126:33751): Couldn't send request to peer 57280ef1740948289e04fb0a6dd73669. Status: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 00:34:51.001520 23387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ecdfb3b4e0f4487873207bd5c590384. Latest consensus state: current_term: 3 leader_uuid: "5ecdfb3b4e0f4487873207bd5c590384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:51.001798 23387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:34:51.005090 23382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "5ecdfb3b4e0f4487873207bd5c590384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:51.005497 23382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:51.006099 23387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "5ecdfb3b4e0f4487873207bd5c590384" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "57280ef1740948289e04fb0a6dd73669" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33751 } } peers { permanent_uuid: "5ecdfb3b4e0f4487873207bd5c590384" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 40257 } attrs { promote: false } } peers { permanent_uuid: "901445618aa9460191789ee586194f48" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42903 } attrs { promote: false } } }
I20260812 00:34:51.006389 23387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 901445618aa9460191789ee586194f48 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:34:51.034700 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.017806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34780 (local address 127.7.244.67:44261)
0812 00:34:51.027000 (+ 9194us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:51.027007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:51.027030 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:34:51.027120 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:51.027127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:51.027209 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:51.027318 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:51.028344 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.029509 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.032217 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.033502 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.033660 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.033771 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:51.034171 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 00:34:51.034343 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9043,"thread_start_us":8896,"threads_started":1}
I20260812 00:34:51.035411 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.017281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:44261 (local address 127.0.0.1:34780)
0812 00:34:51.026264 (+ 8983us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.026291 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:51.026455 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:51.027588 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:51.027594 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:51.027629 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:51.028022 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:51.028040 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.029660 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.029666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:51.032001 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:51.032016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.033740 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.033747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:51.033958 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.033997 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 00:34:51.034100 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 00:34:51.034648 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8807,"mutex_wait_us":1,"thread_start_us":8659,"threads_started":1}
I20260812 00:34:51.036549 23076 tablet_service.cc:1467] Processing CreateTablet for tablet 0c078f9384d84033a256740d7dddbf26 (DEFAULT_TABLE table=test-table [id=facf6f43d2924427b7c570a7395f6e74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:51.037055 23076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c078f9384d84033a256740d7dddbf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:51.037318 23397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.014993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44493 (local address 127.0.0.1:34734)
0812 00:34:51.022101 (+ 7108us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.022122 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:51.022252 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:51.030678 (+ 8426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:51.030683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:51.030713 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:51.031196 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:51.031210 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.032617 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.032622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:51.034710 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:51.034726 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.036548 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.036555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:51.036736 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.036773 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 00:34:51.036873 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 00:34:51.036987 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6971,"mutex_wait_us":2,"thread_start_us":6787,"threads_started":1}
I20260812 00:34:51.037489 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.015176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.7.244.65:44493)
0812 00:34:51.030029 (+ 14853us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:51.030036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:51.030057 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:34:51.030129 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:51.030134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:51.030217 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:51.030323 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:51.031437 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.032435 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.034934 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.036132 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.036272 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.036395 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:51.036961 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 00:34:51.037134 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14694,"thread_start_us":14571,"threads_started":1}
I20260812 00:34:51.038302 22527 tablet_service.cc:1467] Processing CreateTablet for tablet 0c078f9384d84033a256740d7dddbf26 (DEFAULT_TABLE table=test-table [id=facf6f43d2924427b7c570a7395f6e74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:51.038787 22527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c078f9384d84033a256740d7dddbf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:51.039433 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.014617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:45217 (local address 127.0.0.1:41004)
0812 00:34:51.015297 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.015318 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:51.015494 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:51.030970 (+ 15476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:51.030976 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:51.031003 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:51.031411 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:51.031425 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.032799 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.032804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:51.034715 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:51.034730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.038648 (+ 3918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.038655 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:51.038848 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.038885 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 00:34:51.038997 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 00:34:51.039148 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":78}
I20260812 00:34:51.039696 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.014742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.7.244.66:45217)
0812 00:34:51.030346 (+ 15604us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:51.030352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:51.030374 (+ 22us) server_negotiation.cc:407] Connection header received
0812 00:34:51.030565 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:51.030572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:51.030656 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:51.030927 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:51.031601 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.032625 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.034926 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.038305 (+ 3379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.038472 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.038583 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:51.039070 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 00:34:51.039228 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15461,"thread_start_us":16020,"threads_started":1}
I20260812 00:34:51.040521 22803 tablet_service.cc:1467] Processing CreateTablet for tablet 0c078f9384d84033a256740d7dddbf26 (DEFAULT_TABLE table=test-table [id=facf6f43d2924427b7c570a7395f6e74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:51.040987 22803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c078f9384d84033a256740d7dddbf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:51.051985 23409 tablet_bootstrap.cc:492] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: Bootstrap starting.
I20260812 00:34:51.054272 23408 tablet_bootstrap.cc:492] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: Bootstrap starting.
I20260812 00:34:51.054418 23409 tablet_bootstrap.cc:654] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:51.055791 23408 tablet_bootstrap.cc:654] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:51.057492 23410 tablet_bootstrap.cc:492] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: Bootstrap starting.
I20260812 00:34:51.059042 23410 tablet_bootstrap.cc:654] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:51.061425 23408 tablet_bootstrap.cc:492] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: No bootstrap required, opened a new log
I20260812 00:34:51.061618 23410 tablet_bootstrap.cc:492] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: No bootstrap required, opened a new log
I20260812 00:34:51.061625 23408 ts_tablet_manager.cc:1397] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260812 00:34:51.061925 23410 ts_tablet_manager.cc:1397] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260812 00:34:51.061983 23409 tablet_bootstrap.cc:492] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: No bootstrap required, opened a new log
I20260812 00:34:51.062140 23409 ts_tablet_manager.cc:1397] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260812 00:34:51.062565 23408 raft_consensus.cc:348] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.062839 23408 raft_consensus.cc:374] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:51.062716 23410 raft_consensus.cc:348] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.062949 23408 raft_consensus.cc:729] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e591840d9594679926354c22614d745, State: Initialized, Role: FOLLOWER
I20260812 00:34:51.062954 23410 raft_consensus.cc:374] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:51.063057 23410 raft_consensus.cc:729] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e9e1797474436ab2a79319877c53dd, State: Initialized, Role: FOLLOWER
I20260812 00:34:51.062987 23409 raft_consensus.cc:348] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.063215 23409 raft_consensus.cc:374] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:51.063318 23409 raft_consensus.cc:729] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e640fd14b3b1411fa515c1cdbd2d113d, State: Initialized, Role: FOLLOWER
I20260812 00:34:51.063251 23408 consensus_queue.cc:260] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.063310 23410 consensus_queue.cc:260] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.063571 23409 consensus_queue.cc:260] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.064314 23408 ts_tablet_manager.cc:1428] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:34:51.065538 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.065017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:56671)
0812 00:34:51.065221 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.065383 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:34:51.067780 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.067305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:52659)
0812 00:34:51.067483 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.067580 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:34:51.069051 23200 heartbeater.cc:503] Master 127.7.244.125:40257 was elected leader, sending a full tablet report...
I20260812 00:34:51.069985 23409 ts_tablet_manager.cc:1428] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 00:34:51.071568 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.071103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:51015)
0812 00:34:51.071288 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.071407 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 00:34:51.071921 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.071152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:54185)
0812 00:34:51.071671 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.071774 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404}
I20260812 00:34:51.073112 22670 heartbeater.cc:503] Master 127.7.244.125:40257 was elected leader, sending a full tablet report...
I20260812 00:34:51.073577 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.073026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:55329)
0812 00:34:51.073375 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.073442 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234}
I20260812 00:34:51.074641 23410 ts_tablet_manager.cc:1428] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260812 00:34:51.074997 22952 heartbeater.cc:503] Master 127.7.244.125:40257 was elected leader, sending a full tablet report...
I20260812 00:34:51.076309 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.075791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:37995)
0812 00:34:51.076027 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.076143 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260812 00:34:51.153223 23422 raft_consensus.cc:482] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:51.153563 23422 raft_consensus.cc:504] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.154543 23422 leader_election.cc:288] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217), 1e591840d9594679926354c22614d745 (127.7.244.67:44261)
I20260812 00:34:51.155454 22851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c078f9384d84033a256740d7dddbf26" candidate_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e9e1797474436ab2a79319877c53dd" is_pre_election: true
I20260812 00:34:51.155454 23113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c078f9384d84033a256740d7dddbf26" candidate_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e591840d9594679926354c22614d745" is_pre_election: true
I20260812 00:34:51.155758 22851 raft_consensus.cc:2454] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e640fd14b3b1411fa515c1cdbd2d113d in term 0.
I20260812 00:34:51.155759 23113 raft_consensus.cc:2454] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e640fd14b3b1411fa515c1cdbd2d113d in term 0.
I20260812 00:34:51.156483 22435 leader_election.cc:302] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1e591840d9594679926354c22614d745, 94e9e1797474436ab2a79319877c53dd, e640fd14b3b1411fa515c1cdbd2d113d; no voters:
I20260812 00:34:51.156898 23422 raft_consensus.cc:2781] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:51.157038 23422 raft_consensus.cc:482] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:51.157133 23422 raft_consensus.cc:3037] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:51.159675 23422 raft_consensus.cc:504] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.160799 23422 leader_election.cc:288] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [CANDIDATE]: Term 1 election: Requested vote from peers 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217), 1e591840d9594679926354c22614d745 (127.7.244.67:44261)
I20260812 00:34:51.161494 23113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c078f9384d84033a256740d7dddbf26" candidate_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e591840d9594679926354c22614d745"
I20260812 00:34:51.161505 22851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c078f9384d84033a256740d7dddbf26" candidate_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94e9e1797474436ab2a79319877c53dd"
I20260812 00:34:51.161769 22851 raft_consensus.cc:3037] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:51.161754 23113 raft_consensus.cc:3037] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:51.164208 23113 raft_consensus.cc:2454] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e640fd14b3b1411fa515c1cdbd2d113d in term 1.
I20260812 00:34:51.164258 22851 raft_consensus.cc:2454] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e640fd14b3b1411fa515c1cdbd2d113d in term 1.
I20260812 00:34:51.164716 22432 leader_election.cc:302] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e591840d9594679926354c22614d745, e640fd14b3b1411fa515c1cdbd2d113d; no voters:
I20260812 00:34:51.165060 23422 raft_consensus.cc:2781] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:51.165570 23422 raft_consensus.cc:686] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 1 LEADER]: Becoming Leader. State: Replica: e640fd14b3b1411fa515c1cdbd2d113d, State: Running, Role: LEADER
I20260812 00:34:51.166131 23422 consensus_queue.cc:237] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } }
I20260812 00:34:51.166698 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.166203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:44315)
0812 00:34:51.166411 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.166526 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:34:51.169624 21931 catalog_manager.cc:5495] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d reported cstate change: term changed from 0 to 1, leader changed from <none> to e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65). New cstate: current_term: 1 leader_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e640fd14b3b1411fa515c1cdbd2d113d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 } health_report { overall_health: UNKNOWN } } }
I20260812 00:34:51.204712 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.200206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.7.244.65:44493)
0812 00:34:51.200427 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:51.200433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:51.200454 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:34:51.200632 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:51.200636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:51.200728 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:51.200804 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:51.201141 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.201882 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.203215 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.203681 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.203788 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.204153 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:51.204260 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:51.204327 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:51.204429 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 00:34:51.204499 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:34:51.208293 22851 raft_consensus.cc:1261] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Refusing update from remote peer e640fd14b3b1411fa515c1cdbd2d113d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:51.208312 23113 raft_consensus.cc:1261] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 1 FOLLOWER]: Refusing update from remote peer e640fd14b3b1411fa515c1cdbd2d113d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:51.209034 23424 consensus_queue.cc:1035] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [LEADER]: Connected to new peer: Peer: permanent_uuid: "94e9e1797474436ab2a79319877c53dd" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 45217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:51.209385 23422 consensus_queue.cc:1035] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e591840d9594679926354c22614d745" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:51.212311 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.211717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:48211)
0812 00:34:51.211976 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.212100 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260812 00:34:51.212386 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.211843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:44141)
0812 00:34:51.212092 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.212191 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260812 00:34:51.219254 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.218640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.66:40385)
0812 00:34:51.218892 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.219008 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260812 00:34:51.227272 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.226541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:41283)
0812 00:34:51.226916 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.227028 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
I20260812 00:34:51.230995 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.230519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.65:34509)
0812 00:34:51.230708 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.230808 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 00:34:51.236179 21931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:51.236441 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.235640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.7.244.67:37113)
0812 00:34:51.236132 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.236245 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327}
I20260812 00:34:51.236761 21931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:51.243687 21931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5ecdfb3b4e0f4487873207bd5c590384: Sending DeleteTablet for 3 replicas of tablet 0c078f9384d84033a256740d7dddbf26
I20260812 00:34:51.244753 22803 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c078f9384d84033a256740d7dddbf26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:51 UTC) from {username='oryx', principal='oryx/127.7.244.125@KRBTEST.COM'} at 127.0.0.1:41004
I20260812 00:34:51.244832 22527 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c078f9384d84033a256740d7dddbf26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:51 UTC) from {username='oryx', principal='oryx/127.7.244.125@KRBTEST.COM'} at 127.0.0.1:34734
I20260812 00:34:51.245326 23076 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c078f9384d84033a256740d7dddbf26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:51 UTC) from {username='oryx', principal='oryx/127.7.244.125@KRBTEST.COM'} at 127.0.0.1:34780
I20260812 00:34:51.245991 23075 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c078f9384d84033a256740d7dddbf26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:51 UTC) from {username='oryx', principal='oryx/127.7.244.125@KRBTEST.COM'} at 127.0.0.1:34780
I20260812 00:34:51.247531 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 22237
I20260812 00:34:51.258241 23446 tablet_replica.cc:330] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: stopping tablet replica
I20260812 00:34:51.258546 23446 raft_consensus.cc:2229] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:51.258921 23446 raft_consensus.cc:2258] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:51.260244 23446 ts_tablet_manager.cc:1905] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:51.260838 23445 tablet_replica.cc:330] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: stopping tablet replica
I20260812 00:34:51.261175 23445 raft_consensus.cc:2229] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:51.261729 23445 raft_consensus.cc:2258] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:51.263011 23445 ts_tablet_manager.cc:1905] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:51.265719 23443 tablet_replica.cc:330] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: stopping tablet replica
I20260812 00:34:51.265983 23443 raft_consensus.cc:2229] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:51.266337 23443 raft_consensus.cc:2258] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:51.268172 23443 ts_tablet_manager.cc:1905] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:51.268412 23446 ts_tablet_manager.cc:1918] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:51.268568 23446 log.cc:1196] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/0c078f9384d84033a256740d7dddbf26
I20260812 00:34:51.269025 23446 ts_tablet_manager.cc:1939] T 0c078f9384d84033a256740d7dddbf26 P 1e591840d9594679926354c22614d745: Deleting consensus metadata
I20260812 00:34:51.269845 23445 ts_tablet_manager.cc:1918] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:51.270010 23445 log.cc:1196] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/0c078f9384d84033a256740d7dddbf26
I20260812 00:34:51.270340 21910 catalog_manager.cc:4841] TS 1e591840d9594679926354c22614d745 (127.7.244.67:44261): tablet 0c078f9384d84033a256740d7dddbf26 (table test-table [id=facf6f43d2924427b7c570a7395f6e74]) successfully deleted
I20260812 00:34:51.270466 23445 ts_tablet_manager.cc:1939] T 0c078f9384d84033a256740d7dddbf26 P e640fd14b3b1411fa515c1cdbd2d113d: Deleting consensus metadata
W20260812 00:34:51.271492 21910 catalog_manager.cc:4808] TS 1e591840d9594679926354c22614d745 (127.7.244.67:44261): delete failed for tablet 0c078f9384d84033a256740d7dddbf26 because the tablet was not found. No further retry: Not found: Tablet not found: 0c078f9384d84033a256740d7dddbf26
I20260812 00:34:51.271881 21914 catalog_manager.cc:4841] TS e640fd14b3b1411fa515c1cdbd2d113d (127.7.244.65:44493): tablet 0c078f9384d84033a256740d7dddbf26 (table test-table [id=facf6f43d2924427b7c570a7395f6e74]) successfully deleted
I20260812 00:34:51.274257 23443 ts_tablet_manager.cc:1918] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:51.274416 23443 log.cc:1196] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/0c078f9384d84033a256740d7dddbf26
I20260812 00:34:51.274863 23443 ts_tablet_manager.cc:1939] T 0c078f9384d84033a256740d7dddbf26 P 94e9e1797474436ab2a79319877c53dd: Deleting consensus metadata
I20260812 00:34:51.275857 21914 catalog_manager.cc:4841] TS 94e9e1797474436ab2a79319877c53dd (127.7.244.66:45217): tablet 0c078f9384d84033a256740d7dddbf26 (table test-table [id=facf6f43d2924427b7c570a7395f6e74]) successfully deleted
I20260812 00:34:51.285646 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.248507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.7.244.65:44493)
0812 00:34:51.248698 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:51.248704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:51.262437 (+ 13733us) server_negotiation.cc:407] Connection header received
0812 00:34:51.265796 (+ 3359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:51.265802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:51.265949 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:51.266059 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:51.268473 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.269437 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.275165 (+ 5728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.275830 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.275992 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.278665 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:51.278689 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:51.278712 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:51.278765 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:51.282524 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:51.283312 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:51.283322 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:51.283326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:51.283517 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:51.283892 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:51.283897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:51.283901 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:51.284394 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:51.284707 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:51.284997 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 00:34:51.285235 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:34:51.286406 22654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:51.421576 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 22237
I20260812 00:34:51.459757 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 22685
I20260812 00:34:51.474406 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.460582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41020 (local address 127.7.244.66:45217)
0812 00:34:51.460762 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:51.460766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:51.460786 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:34:51.460925 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:51.460929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:51.460996 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:51.461100 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:51.462581 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.463337 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.464758 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.465212 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.465454 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.468191 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:51.468214 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:51.468234 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:51.468278 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:51.471272 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:51.472110 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:51.472119 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:51.472121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:51.472275 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:51.472673 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:51.472678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:51.472681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:51.473119 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:51.473477 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:51.473904 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 00:34:51.474111 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 00:34:51.475159 22941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:51.483016 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.482448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:38310)
0812 00:34:51.482651 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.482796 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:34:51.484591 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.484080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33751 (local address 127.0.0.1:38322)
0812 00:34:51.484253 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:51.484370 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:33751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 00:34:51.571698 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 22685
I20260812 00:34:51.601432 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 22971
I20260812 00:34:51.615998 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:51.602267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.7.244.67:44261)
0812 00:34:51.602758 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:51.602764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:51.602782 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:34:51.602844 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:51.602847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:51.602914 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:51.603020 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:51.604493 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.605191 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.606610 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:51.607027 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:51.607131 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:51.610058 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:51.610078 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:51.610096 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:51.610136 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:51.613154 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:51.613953 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:51.613962 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:51.613964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:51.614149 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:51.614504 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:51.614508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:51.614510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:51.614909 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:51.615185 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:51.615450 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 00:34:51.615642 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":267,"threads_started":1}
I20260812 00:34:51.616711 23188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:51.712502 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 22971
I20260812 00:34:51.745714 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21673
I20260812 00:34:51.746825 21997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:51.871158 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21673
I20260812 00:34:51.905788 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21867
I20260812 00:34:51.907033 22347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:52.025967 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21867
2026-08-12T00:34:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12395 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:34:52 dist-test-slave-gjw7 krb5kdc[23466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:34:52 dist-test-slave-gjw7 krb5kdc[23466](info): set up 2 sockets
Aug 12 00:34:52 dist-test-slave-gjw7 krb5kdc[23466](info): commencing operation
krb5kdc: starting...
W20260812 00:34:53.371258 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:34:53 dist-test-slave-gjw7 krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:34:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:34:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.126.keytab.
Entry for principal oryx/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.126.keytab.
I20260812 00:34:53.645360 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33135
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.126.keytab
--principal=oryx/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:33135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:54.002579 23484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:54.002988 23484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:54.003149 23484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:54.010957 23484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:34:54.011106 23484 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 00:34:54.011148 23484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:54.011220 23484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:54.011299 23484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:34:54.011382 23484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:34:54.022075 23484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:33135
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.126.keytab
--principal=oryx/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:54.023537 23484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:54.025018 23484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:54.038455 23501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:54.038522 23497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:54.038501 23498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:54.039382 23484 server_base.cc:1034] running on GCE node
I20260812 00:34:54.040338 23484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:54.041934 23484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:54.043172 23484 hybrid_clock.cc:648] HybridClock initialized: now 1786494894043107 us; error 33 us; skew 500 ppm
I20260812 00:34:54.043545 23484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:54 dist-test-slave-gjw7 krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:54.053679 23484 init.cc:377] Logged in from keytab as oryx/127.7.244.126@KRBTEST.COM (short username oryx)
I20260812 00:34:54.055569 23484 webserver.cc:460] Webserver started at http://127.7.244.126:36425/ using document root <none> and password file <none>
I20260812 00:34:54.056403 23484 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:54.056517 23484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:54.056872 23484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:54.060266 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "651b0b62d8d94975bfd9b25938dbba80"
format_stamp: "Formatted at 2026-08-12 00:34:54 on dist-test-slave-gjw7"
server_key: "ac2ee66d4e97e1937ee0f4a6d436814f"
server_key_iv: "847499891a77e4735a24be30af77cd32"
server_key_version: "encryptionkey@0"
I20260812 00:34:54.061208 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "651b0b62d8d94975bfd9b25938dbba80"
format_stamp: "Formatted at 2026-08-12 00:34:54 on dist-test-slave-gjw7"
server_key: "ac2ee66d4e97e1937ee0f4a6d436814f"
server_key_iv: "847499891a77e4735a24be30af77cd32"
server_key_version: "encryptionkey@0"
I20260812 00:34:54.067159 23484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 00:34:54.071548 23519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:54.073148 23484 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 00:34:54.073408 23484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "651b0b62d8d94975bfd9b25938dbba80"
format_stamp: "Formatted at 2026-08-12 00:34:54 on dist-test-slave-gjw7"
server_key: "ac2ee66d4e97e1937ee0f4a6d436814f"
server_key_iv: "847499891a77e4735a24be30af77cd32"
server_key_version: "encryptionkey@0"
I20260812 00:34:54.073661 23484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:54.094339 23484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:54.099057 23484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:54.099339 23484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:54.122021 23484 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:33135
I20260812 00:34:54.122045 23650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:33135 every 8 connection(s)
I20260812 00:34:54.123593 23484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:34:54.127985 23651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:54.133589 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 23484
I20260812 00:34:54.133831 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:34:54.134658 8145 external_mini_cluster.cc:1428] Setting key 8604cc4764bdcbb954cade8cfe1cab65
I20260812 00:34:54.140933 23651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: Bootstrap starting.
I20260812 00:34:54.145037 23651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:54.146420 23651 log.cc:824] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:54.150197 23651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: No bootstrap required, opened a new log
Aug 12 00:34:54 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:54.159196 23651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651b0b62d8d94975bfd9b25938dbba80" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33135 } }
I20260812 00:34:54.159654 23651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:54.159772 23651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 651b0b62d8d94975bfd9b25938dbba80, State: Initialized, Role: FOLLOWER
I20260812 00:34:54.160521 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:54.136985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60166 (local address 127.7.244.126:33135)
0812 00:34:54.137862 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:54.137875 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:54.137938 (+ 63us) server_negotiation.cc:407] Connection header received
0812 00:34:54.139848 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:54.139878 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:54.140367 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:54.140759 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:54.142841 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.144253 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.146029 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.147222 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.147425 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:54.151863 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:54.151872 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:54.151891 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:54.151941 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:54.155550 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:54.156445 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:54.156453 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:54.156461 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:54.156692 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:54.157149 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:54.157157 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:54.157159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:54.157896 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:54.158228 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:54.158741 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 00:34:54.159222 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":383,"threads_started":1}
I20260812 00:34:54.160413 23651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651b0b62d8d94975bfd9b25938dbba80" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33135 } }
I20260812 00:34:54.160684 23651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:34:54.160822 23651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:34:54.161008 23651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:54.163560 23651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651b0b62d8d94975bfd9b25938dbba80" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33135 } }
I20260812 00:34:54.164155 23651 leader_election.cc:302] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 651b0b62d8d94975bfd9b25938dbba80; no voters:
I20260812 00:34:54.164723 23651 leader_election.cc:288] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:34:54.164866 23661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:54.165576 23661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [term 1 LEADER]: Becoming Leader. State: Replica: 651b0b62d8d94975bfd9b25938dbba80, State: Running, Role: LEADER
I20260812 00:34:54.166157 23661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651b0b62d8d94975bfd9b25938dbba80" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33135 } }
I20260812 00:34:54.166853 23651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:34:54.169049 23662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "651b0b62d8d94975bfd9b25938dbba80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651b0b62d8d94975bfd9b25938dbba80" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33135 } } }
I20260812 00:34:54.169516 23662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:54.171598 23680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:34:54.171537 23671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 651b0b62d8d94975bfd9b25938dbba80. Latest consensus state: current_term: 1 leader_uuid: "651b0b62d8d94975bfd9b25938dbba80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "651b0b62d8d94975bfd9b25938dbba80" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33135 } } }
I20260812 00:34:54.171890 23671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80 [sys.catalog]: This master's current role is: LEADER
I20260812 00:34:54.177526 23680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:34:54.189145 23680 catalog_manager.cc:1347] Generated new cluster ID: d34c3a5c7ba3451b8374c4e140c02375
I20260812 00:34:54.189321 23680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:34:54.203058 23680 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:34:54.204432 23680 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:34:54.212677 23680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: Generated new TSK 0
I20260812 00:34:54.213677 23680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab.
Entry for principal oryx/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab.
I20260812 00:34:54.288388 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33135
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab
--principal=oryx/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:54.683713 23718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:54.684041 23718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:54.684140 23718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:54.691606 23718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:54.691752 23718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:54.691848 23718 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:34:54.691938 23718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:34:54.702875 23718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab
--principal=oryx/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:33135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:54.704524 23718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:54.706004 23718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:54.719792 23729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:54.719760 23737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:54.719849 23730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:54.720655 23718 server_base.cc:1034] running on GCE node
I20260812 00:34:54.721388 23718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:54.722419 23718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:54.723649 23718 hybrid_clock.cc:648] HybridClock initialized: now 1786494894723577 us; error 46 us; skew 500 ppm
I20260812 00:34:54.723982 23718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:54 dist-test-slave-gjw7 krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:54.727371 23718 init.cc:377] Logged in from keytab as oryx/127.7.244.65@KRBTEST.COM (short username oryx)
I20260812 00:34:54.728915 23718 webserver.cc:460] Webserver started at http://127.7.244.65:45417/ using document root <none> and password file <none>
I20260812 00:34:54.729736 23718 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:54.729838 23718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:54.730146 23718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:54.733332 23718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "7e715bfb5c8848df92d3ce23d8c1d675"
format_stamp: "Formatted at 2026-08-12 00:34:54 on dist-test-slave-gjw7"
server_key: "07487c684b172f64e7be610abef2569c"
server_key_iv: "14ed7bd3b0e5448d91686f76bb2f63c9"
server_key_version: "encryptionkey@0"
I20260812 00:34:54.734194 23718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "7e715bfb5c8848df92d3ce23d8c1d675"
format_stamp: "Formatted at 2026-08-12 00:34:54 on dist-test-slave-gjw7"
server_key: "07487c684b172f64e7be610abef2569c"
server_key_iv: "14ed7bd3b0e5448d91686f76bb2f63c9"
server_key_version: "encryptionkey@0"
I20260812 00:34:54.740279 23718 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260812 00:34:54.744742 23746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:54.746356 23718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 00:34:54.746546 23718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "7e715bfb5c8848df92d3ce23d8c1d675"
format_stamp: "Formatted at 2026-08-12 00:34:54 on dist-test-slave-gjw7"
server_key: "07487c684b172f64e7be610abef2569c"
server_key_iv: "14ed7bd3b0e5448d91686f76bb2f63c9"
server_key_version: "encryptionkey@0"
I20260812 00:34:54.746800 23718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:54.769784 23718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:54.774408 23718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:54.774698 23718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:54.777802 23718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:54.777944 23718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:54.778136 23718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:54.778210 23718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:34:54 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:54.805068 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:54.778582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:49569 (local address 127.7.244.126:33135)
0812 00:34:54.779106 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:54.779114 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:54.780711 (+ 1597us) server_negotiation.cc:407] Connection header received
0812 00:34:54.783177 (+ 2466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:54.783184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:54.783292 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:54.783409 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:54.785858 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.786863 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.790428 (+ 3565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.790971 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.791106 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:54.797185 (+ 6079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:54.797195 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:54.797200 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:54.797257 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:54.800841 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:54.802259 (+ 1418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:54.802264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:54.802266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:54.802422 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:54.803020 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:54.803023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:54.803025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:54.803312 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:54.803430 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:54.804568 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0812 00:34:54.804794 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":264,"threads_started":1}
I20260812 00:34:54.807006 23764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:54.779500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33135 (local address 127.7.244.65:49569)
0812 00:34:54.780501 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:54.780561 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:54.782795 (+ 2234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:54.783749 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:54.783760 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:54.784600 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:54.785501 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:54.785537 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.787112 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.787123 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:54.790066 (+ 2943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:54.790098 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.791261 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.791271 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:54.791701 (+ 430us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:54.792700 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:54.792744 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:54.796754 (+ 4010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:54.801081 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:54.801095 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:54.801120 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:54.802051 (+ 931us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:54.802540 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:54.802545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:54.802550 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:54.802909 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:54.803437 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:54.803447 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:54.804039 (+ 592us) client_negotiation.cc:769] Sending connection context
0812 00:34:54.804404 (+ 365us) client_negotiation.cc:240] Negotiation successful
0812 00:34:54.805090 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":452,"threads_started":1}
I20260812 00:34:54.828768 23760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:54.813821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:34579 (local address 127.7.244.126:33135)
0812 00:34:54.814029 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:54.814036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:54.814800 (+ 764us) server_negotiation.cc:407] Connection header received
0812 00:34:54.814990 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:54.814996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:54.815072 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:54.815159 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:54.816207 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.816945 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.818954 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.819569 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.819703 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:54.822201 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:54.822209 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:54.822214 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:54.822261 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:54.825430 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:54.826460 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:54.826467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:54.826471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:54.826654 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:54.827110 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:54.827116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:54.827119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:54.827487 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:54.827621 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:54.828213 (+ 592us) server_negotiation.cc:299] Negotiation successful
0812 00:34:54.828514 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:34:54.829066 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:54.813875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33135 (local address 127.7.244.65:34579)
0812 00:34:54.814666 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:54.814687 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:54.814860 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:54.815205 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:54.815210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:54.815721 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:54.816038 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:54.816049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.817100 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.817106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:54.818765 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:54.818788 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:54.819802 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:54.819815 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:54.820001 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:54.820792 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:54.820818 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:54.821957 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:54.825635 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:54.825641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:54.825645 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:54.826265 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:54.826774 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:54.826780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:54.826784 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:54.826942 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:54.827626 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:54.827631 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:54.827877 (+ 246us) client_negotiation.cc:769] Sending connection context
0812 00:34:54.828104 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 00:34:54.828557 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":455,"threads_started":1}
I20260812 00:34:54.830338 23718 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:38439
I20260812 00:34:54.830356 23984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:38439 every 8 connection(s)
I20260812 00:34:54.831779 23718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:34:54.834153 23987 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33135
I20260812 00:34:54.834388 23987 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:54.835027 23987 heartbeater.cc:511] Master 127.7.244.126:33135 requested a full tablet report, sending...
I20260812 00:34:54.837455 23556 ts_manager.cc:194] Registered new tserver with Master: 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439)
I20260812 00:34:54.839006 23556 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:49569
I20260812 00:34:54.842227 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 23718
I20260812 00:34:54.842412 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:34:54.843096 8145 external_mini_cluster.cc:1428] Setting key 2d625642613d054ecd944b2094d87cb6
WARNING: no policy specified for oryx/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab.
Entry for principal oryx/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab.
I20260812 00:34:54.901054 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33135
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab
--principal=oryx/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:34:55.253275 23994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:55.253832 23994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:55.253998 23994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:55.261749 23994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:55.261907 23994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:55.262039 23994 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:34:55.262156 23994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:34:55.272503 23994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab
--principal=oryx/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:33135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:55.273977 23994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:55.275399 23994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:55.289731 24003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:55.289796 24004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:55.289777 24008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:55.290789 23994 server_base.cc:1034] running on GCE node
I20260812 00:34:55.291528 23994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:55.292572 23994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:55.293818 23994 hybrid_clock.cc:648] HybridClock initialized: now 1786494895293738 us; error 52 us; skew 500 ppm
I20260812 00:34:55.294168 23994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:55 dist-test-slave-gjw7 krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494895, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:55.297742 23994 init.cc:377] Logged in from keytab as oryx/127.7.244.66@KRBTEST.COM (short username oryx)
I20260812 00:34:55.299468 23994 webserver.cc:460] Webserver started at http://127.7.244.66:39003/ using document root <none> and password file <none>
I20260812 00:34:55.300242 23994 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:55.300359 23994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:55.300710 23994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:55.304052 23994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "506d7dfe91af4e75bdde8c8a73fedd11"
format_stamp: "Formatted at 2026-08-12 00:34:55 on dist-test-slave-gjw7"
server_key: "b447a87200a78f802039e457e431554e"
server_key_iv: "dad144326781e1e1dd14ab9a7493006c"
server_key_version: "encryptionkey@0"
I20260812 00:34:55.304960 23994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "506d7dfe91af4e75bdde8c8a73fedd11"
format_stamp: "Formatted at 2026-08-12 00:34:55 on dist-test-slave-gjw7"
server_key: "b447a87200a78f802039e457e431554e"
server_key_iv: "dad144326781e1e1dd14ab9a7493006c"
server_key_version: "encryptionkey@0"
I20260812 00:34:55.310971 23994 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 00:34:55.315141 24032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:55.316766 23994 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 00:34:55.316974 23994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "506d7dfe91af4e75bdde8c8a73fedd11"
format_stamp: "Formatted at 2026-08-12 00:34:55 on dist-test-slave-gjw7"
server_key: "b447a87200a78f802039e457e431554e"
server_key_iv: "dad144326781e1e1dd14ab9a7493006c"
server_key_version: "encryptionkey@0"
I20260812 00:34:55.317224 23994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:55.333173 23994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:55.337857 23994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:55.338152 23994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:55.341045 23994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:55.341161 23994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:55.341339 23994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:55.341441 23994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 12 00:34:55 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494895, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:55.367183 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:55.341610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:57737 (local address 127.7.244.126:33135)
0812 00:34:55.342414 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:55.342419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:55.343459 (+ 1040us) server_negotiation.cc:407] Connection header received
0812 00:34:55.345653 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:55.345659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:55.345747 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:55.345849 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:55.348505 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:55.349566 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:55.352827 (+ 3261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:55.353427 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:55.353579 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:55.359184 (+ 5605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:55.359193 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:55.359198 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:55.359250 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:55.362406 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:55.363782 (+ 1376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:55.363789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:55.363793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:55.364012 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:55.364700 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:55.364706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:55.364710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:55.365031 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:55.365188 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:55.366568 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0812 00:34:55.366883 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":589,"threads_started":1}
I20260812 00:34:55.369050 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:55.342390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33135 (local address 127.7.244.66:57737)
0812 00:34:55.343258 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:55.343309 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:55.345325 (+ 2016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:55.346185 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:55.346199 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:55.347225 (+ 1026us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:55.348250 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:55.348279 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:55.349791 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:55.349802 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:55.352555 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:55.352591 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:55.353935 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:55.353945 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:55.354253 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:55.355088 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:55.355131 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:55.358808 (+ 3677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:55.362619 (+ 3811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:55.362634 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:55.362660 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:55.363545 (+ 885us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:55.364181 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:55.364188 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:55.364191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:55.364555 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:55.365252 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:55.365262 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:55.365934 (+ 672us) client_negotiation.cc:769] Sending connection context
0812 00:34:55.366393 (+ 459us) client_negotiation.cc:240] Negotiation successful
0812 00:34:55.367148 (+ 755us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":393,"threads_started":1}
I20260812 00:34:55.398141 23994 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:41907
I20260812 00:34:55.398186 24264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:41907 every 8 connection(s)
I20260812 00:34:55.401646 23994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:34:55.405534 24267 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33135
I20260812 00:34:55.405967 24267 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:55.407218 24267 heartbeater.cc:511] Master 127.7.244.126:33135 requested a full tablet report, sending...
I20260812 00:34:55.408813 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 23994
I20260812 00:34:55.409112 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:34:55.410230 8145 external_mini_cluster.cc:1428] Setting key 9e6d82582a8da5aa0a13ce7dce1b7f64
I20260812 00:34:55.410722 23556 ts_manager.cc:194] Registered new tserver with Master: 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907)
I20260812 00:34:55.412565 23556 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:57737
I20260812 00:34:55.413585 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:55.381664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:50171 (local address 127.7.244.126:33135)
0812 00:34:55.382002 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:55.382020 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:55.385494 (+ 3474us) server_negotiation.cc:407] Connection header received
0812 00:34:55.385995 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:55.386014 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:55.386214 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:55.386457 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:55.389353 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:55.391196 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:55.395141 (+ 3945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:55.396219 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:55.396545 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:55.401262 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:55.401277 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:55.401281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:55.401411 (+ 130us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:55.407309 (+ 5898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:55.409084 (+ 1775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:55.409103 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:55.409110 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:55.409512 (+ 402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:55.410476 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:55.410494 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:55.410498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:55.411435 (+ 937us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:55.411658 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:55.412470 (+ 812us) server_negotiation.cc:299] Negotiation successful
0812 00:34:55.412905 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 00:34:55.413735 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:55.380788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33135 (local address 127.7.244.66:50171)
0812 00:34:55.385246 (+ 4458us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:55.385276 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:55.385711 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:55.386520 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:55.386530 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:55.387922 (+ 1392us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:55.388963 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:55.388999 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:55.391563 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:55.391571 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:55.394803 (+ 3232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:55.394848 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:55.396723 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:55.396749 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:55.397229 (+ 480us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:55.398639 (+ 1410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:55.398670 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:55.400834 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:55.407566 (+ 6732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:55.407584 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:55.407600 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:55.408815 (+ 1215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:55.409734 (+ 919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:55.409750 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:55.409754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:55.410158 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:55.411644 (+ 1486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:55.411650 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:55.411945 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 00:34:55.412143 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 00:34:55.412831 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4264,"thread_start_us":4064,"threads_started":1}
WARNING: no policy specified for oryx/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab.
Entry for principal oryx/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab.
I20260812 00:34:55.497963 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33135
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab
--principal=oryx/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:34:55.842406 23987 heartbeater.cc:503] Master 127.7.244.126:33135 was elected leader, sending a full tablet report...
W20260812 00:34:56.128654 24277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:34:56.129213 24277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:34:56.129630 24277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:34:56.144429 24277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:34:56.144649 24277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:34:56.144852 24277 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 00:34:56.145006 24277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:34:56.161568 24277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab
--principal=oryx/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.244.126:33135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:34:56.163589 24277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:34:56.165809 24277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:34:56.188725 24287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:56.188617 24290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:34:56.188687 24288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:34:56.190726 24277 server_base.cc:1034] running on GCE node
I20260812 00:34:56.191730 24277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:34:56.193177 24277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:34:56.194554 24277 hybrid_clock.cc:648] HybridClock initialized: now 1786494896194434 us; error 80 us; skew 500 ppm
I20260812 00:34:56.195050 24277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:34:56 dist-test-slave-gjw7 krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494896, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:34:56.200201 24277 init.cc:377] Logged in from keytab as oryx/127.7.244.67@KRBTEST.COM (short username oryx)
I20260812 00:34:56.202742 24277 webserver.cc:460] Webserver started at http://127.7.244.67:42991/ using document root <none> and password file <none>
I20260812 00:34:56.203861 24277 fs_manager.cc:359] Metadata directory not provided
I20260812 00:34:56.204030 24277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:34:56.204500 24277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:34:56.208961 24277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "6a54d44392da41608e7fcb22095564fe"
format_stamp: "Formatted at 2026-08-12 00:34:56 on dist-test-slave-gjw7"
server_key: "faea5da618ad21c11bb8472b02afae64"
server_key_iv: "38a500380da18a084fe9d9d6035d7106"
server_key_version: "encryptionkey@0"
I20260812 00:34:56.210310 24277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "6a54d44392da41608e7fcb22095564fe"
format_stamp: "Formatted at 2026-08-12 00:34:56 on dist-test-slave-gjw7"
server_key: "faea5da618ad21c11bb8472b02afae64"
server_key_iv: "38a500380da18a084fe9d9d6035d7106"
server_key_version: "encryptionkey@0"
I20260812 00:34:56.218891 24277 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260812 00:34:56.225112 24305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:56.227600 24277 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260812 00:34:56.227873 24277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "6a54d44392da41608e7fcb22095564fe"
format_stamp: "Formatted at 2026-08-12 00:34:56 on dist-test-slave-gjw7"
server_key: "faea5da618ad21c11bb8472b02afae64"
server_key_iv: "38a500380da18a084fe9d9d6035d7106"
server_key_version: "encryptionkey@0"
I20260812 00:34:56.228235 24277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:34:56.274346 24277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:34:56.281594 24277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:34:56.282012 24277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:34:56.286159 24277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:34:56.286332 24277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:34:56.286703 24277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:34:56.286860 24277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:34:56 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494896, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:56.332057 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:56.287340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:49433 (local address 127.7.244.126:33135)
0812 00:34:56.288037 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:56.288046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:56.290622 (+ 2576us) server_negotiation.cc:407] Connection header received
0812 00:34:56.295112 (+ 4490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:56.295122 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:56.295268 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:56.295418 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:56.299111 (+ 3693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.301057 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.307242 (+ 6185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.308689 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.309165 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:56.318406 (+ 9241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:56.318414 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:56.318417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:56.318494 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:56.324628 (+ 6134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:56.326530 (+ 1902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:56.326546 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:56.326550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:56.326919 (+ 369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:56.327915 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:56.327922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:56.327924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:56.328400 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:56.328651 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:56.330562 (+ 1911us) server_negotiation.cc:299] Negotiation successful
0812 00:34:56.331105 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":341,"threads_started":1}
I20260812 00:34:56.335258 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:56.288945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33135 (local address 127.7.244.67:49433)
0812 00:34:56.290340 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:56.290424 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:56.294594 (+ 4170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:56.295834 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:56.295850 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:56.297011 (+ 1161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:56.298769 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:56.298819 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.301534 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.301561 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:56.306746 (+ 5185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:56.306805 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.309458 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.309480 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:56.310437 (+ 957us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:56.311944 (+ 1507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:56.311998 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:56.317865 (+ 5867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:56.325151 (+ 7286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:56.325178 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:56.325208 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:56.326282 (+ 1074us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:56.327252 (+ 970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:56.327257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:56.327269 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:56.327701 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:56.328657 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:56.328683 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:56.329603 (+ 920us) client_negotiation.cc:769] Sending connection context
0812 00:34:56.330328 (+ 725us) client_negotiation.cc:240] Negotiation successful
0812 00:34:56.331663 (+ 1335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"thread_start_us":634,"threads_started":1}
I20260812 00:34:56.367295 24277 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:39241
I20260812 00:34:56.367295 24533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:39241 every 8 connection(s)
I20260812 00:34:56.370039 24277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:34:56.371136 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 24277
I20260812 00:34:56.371557 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:34:56.372720 8145 external_mini_cluster.cc:1428] Setting key d0c0778c32870beb31926d012885844e
I20260812 00:34:56.375788 24536 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33135
I20260812 00:34:56.376539 24536 heartbeater.cc:464] Registering TS with master...
I20260812 00:34:56.380420 24536 heartbeater.cc:511] Master 127.7.244.126:33135 requested a full tablet report, sending...
I20260812 00:34:56.384037 23556 ts_manager.cc:194] Registered new tserver with Master: 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241)
I20260812 00:34:56.384280 24506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:56.349057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33135 (local address 127.7.244.67:54375)
0812 00:34:56.356786 (+ 7729us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:56.356833 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:56.357210 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:56.357892 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:56.357897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:56.358966 (+ 1069us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:56.359938 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:56.359984 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.362338 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.362361 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:56.364683 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:56.364716 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.366630 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.366649 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:56.367051 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:56.368184 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:56.368234 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:56.369778 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:56.378743 (+ 8965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:56.378761 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:56.378783 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:56.380171 (+ 1388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:56.381409 (+ 1238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:56.381417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:56.381421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:56.381741 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:56.382651 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:56.382655 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:56.383008 (+ 353us) client_negotiation.cc:769] Sending connection context
0812 00:34:56.383259 (+ 251us) client_negotiation.cc:240] Negotiation successful
0812 00:34:56.383764 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7390,"thread_start_us":7018,"threads_started":1}
I20260812 00:34:56.385160 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:56.350003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:54375 (local address 127.7.244.126:33135)
0812 00:34:56.350332 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:56.350341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:56.357035 (+ 6694us) server_negotiation.cc:407] Connection header received
0812 00:34:56.357509 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:56.357523 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:56.357654 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:56.357816 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:56.360488 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.362059 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.365000 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.366240 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.366555 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:56.370164 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:56.370172 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:56.370175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:56.370233 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:56.378268 (+ 8035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:56.380583 (+ 2315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:56.380604 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:56.380610 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:56.380968 (+ 358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:56.381968 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:56.381980 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:56.381983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:56.382457 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:56.382775 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:56.383541 (+ 766us) server_negotiation.cc:299] Negotiation successful
0812 00:34:56.384089 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 00:34:56.386314 23556 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:49433
I20260812 00:34:56.394434 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:34:56.419330 24267 heartbeater.cc:503] Master 127.7.244.126:33135 was elected leader, sending a full tablet report...
Aug 12 00:34:56 dist-test-slave-gjw7 krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494896, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 00:34:56 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:34:56 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:34:57.392256 24536 heartbeater.cc:503] Master 127.7.244.126:33135 was elected leader, sending a full tablet report...
W20260812 00:34:57.539754 24556 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.044s user 0.000s sys 0.019s
W20260812 00:34:57.541033 24320 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:56.484531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60170 (local address 127.7.244.126:33135)
0812 00:34:56.484891 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:56.484920 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:56.485229 (+ 309us) server_negotiation.cc:407] Connection header received
0812 00:34:56.485677 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:56.485685 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:56.485795 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:56.486012 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:56.488664 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.490304 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.493247 (+ 2943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:56.494325 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:56.494557 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.540786 (+1046229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60170: BlockingRecv error: recv got EOF from 127.0.0.1:60170 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 12 00:34:57 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494896, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:57.559442 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.545087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.7.244.126:33135)
0812 00:34:57.545282 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:57.545310 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:57.545586 (+ 276us) server_negotiation.cc:407] Connection header received
0812 00:34:57.545787 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:57.545792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:57.545875 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:57.545968 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:57.547256 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.547980 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.549271 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.549772 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.549872 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.553969 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:57.553974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:57.553978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:57.554018 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:57.556662 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:57.557372 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:57.557378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:57.557382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:57.557527 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:57.557919 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:57.557923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:57.557925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:57.558223 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:57.558483 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:57.558832 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 00:34:57.559082 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:34:57.566913 23555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:57.575681 23555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:57.600991 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.590661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52534 (local address 127.7.244.66:41907)
0812 00:34:57.592976 (+ 2315us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:57.592985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:57.593014 (+ 29us) server_negotiation.cc:407] Connection header received
0812 00:34:57.593108 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:57.593115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:57.593404 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:57.593592 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:57.594400 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.595658 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.598248 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.599838 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.599996 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.600205 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:57.600488 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 00:34:57.600684 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2155,"thread_start_us":1259,"threads_started":1}
I20260812 00:34:57.601585 24583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.589219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38439 (local address 127.0.0.1:55430)
0812 00:34:57.591225 (+ 2006us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:57.591274 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:57.591596 (+ 322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:57.592467 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:57.592475 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:57.592508 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:57.592984 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.593000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.595039 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.595045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.597659 (+ 2614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.597672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.599713 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.599720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.599931 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.599989 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 00:34:57.600447 (+ 458us) client_negotiation.cc:240] Negotiation successful
0812 00:34:57.600673 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1697,"mutex_wait_us":75,"thread_start_us":3049,"threads_started":1}
I20260812 00:34:57.601737 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.589491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55430 (local address 127.7.244.65:38439)
0812 00:34:57.590466 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:57.590476 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:57.591412 (+ 936us) server_negotiation.cc:407] Connection header received
0812 00:34:57.591878 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:57.591888 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:57.592169 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:57.593225 (+ 1056us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:57.593491 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.594873 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.597832 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.599451 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.599612 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.599867 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:57.601208 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0812 00:34:57.601440 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":814,"thread_start_us":665,"threads_started":1}
I20260812 00:34:57.602136 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.589879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:41907 (local address 127.0.0.1:52534)
0812 00:34:57.592528 (+ 2649us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:57.592546 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:57.592662 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:57.593683 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:57.593687 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:57.593713 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:57.594218 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.594231 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.595844 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.595849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.598054 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.598073 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.600090 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.600097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.600260 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.600286 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 00:34:57.601244 (+ 958us) client_negotiation.cc:240] Negotiation successful
0812 00:34:57.601437 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2527,"mutex_wait_us":4,"thread_start_us":877,"threads_started":1}
I20260812 00:34:57.602963 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.593619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:39241 (local address 127.0.0.1:43508)
0812 00:34:57.594205 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:57.594223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:57.594357 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:57.594915 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:57.594920 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:57.594947 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:57.595403 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.595415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.597260 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.597266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.599084 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.599099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.602111 (+ 3012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.602134 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.602334 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.602370 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 00:34:57.602473 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 00:34:57.602612 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":3,"thread_start_us":294,"threads_started":1}
I20260812 00:34:57.603132 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.592986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43508 (local address 127.7.244.67:39241)
0812 00:34:57.593594 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:57.593602 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:57.594346 (+ 744us) server_negotiation.cc:407] Connection header received
0812 00:34:57.594504 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:57.594511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:57.594704 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:57.594910 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:57.595584 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.597090 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.599280 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.600674 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.600811 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.601023 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:57.602584 (+ 1561us) server_negotiation.cc:299] Negotiation successful
0812 00:34:57.602767 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":320,"threads_started":1}
I20260812 00:34:57.605970 24128 tablet_service.cc:1467] Processing CreateTablet for tablet c2c79fc1e5b841b6a465485bb61f2ddd (DEFAULT_TABLE table=test-table [id=ef9e9c8bd13b4b059829dfad8c659de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:57.606323 23843 tablet_service.cc:1467] Processing CreateTablet for tablet c2c79fc1e5b841b6a465485bb61f2ddd (DEFAULT_TABLE table=test-table [id=ef9e9c8bd13b4b059829dfad8c659de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:57.606884 24407 tablet_service.cc:1467] Processing CreateTablet for tablet c2c79fc1e5b841b6a465485bb61f2ddd (DEFAULT_TABLE table=test-table [id=ef9e9c8bd13b4b059829dfad8c659de2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:57.607503 24128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c79fc1e5b841b6a465485bb61f2ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:57.607846 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c79fc1e5b841b6a465485bb61f2ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:57.608297 24407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2c79fc1e5b841b6a465485bb61f2ddd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:57.621506 24589 tablet_bootstrap.cc:492] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Bootstrap starting.
I20260812 00:34:57.623468 24590 tablet_bootstrap.cc:492] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: Bootstrap starting.
I20260812 00:34:57.624266 24591 tablet_bootstrap.cc:492] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: Bootstrap starting.
I20260812 00:34:57.625327 24589 tablet_bootstrap.cc:654] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:57.626737 24589 log.cc:824] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:57.627912 24591 tablet_bootstrap.cc:654] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:57.628270 24590 tablet_bootstrap.cc:654] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:57.629340 24591 log.cc:824] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:57.629521 24590 log.cc:824] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: Log is configured to *not* fsync() on all Append() calls
I20260812 00:34:57.631453 24589 tablet_bootstrap.cc:492] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: No bootstrap required, opened a new log
I20260812 00:34:57.631810 24589 ts_tablet_manager.cc:1397] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260812 00:34:57.636041 24590 tablet_bootstrap.cc:492] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: No bootstrap required, opened a new log
I20260812 00:34:57.636340 24590 ts_tablet_manager.cc:1397] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.008s
I20260812 00:34:57.644105 24591 tablet_bootstrap.cc:492] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: No bootstrap required, opened a new log
I20260812 00:34:57.644049 24589 raft_consensus.cc:348] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.644451 24591 ts_tablet_manager.cc:1397] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.013s
I20260812 00:34:57.644599 24589 raft_consensus.cc:374] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:57.644706 24589 raft_consensus.cc:729] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506d7dfe91af4e75bdde8c8a73fedd11, State: Initialized, Role: FOLLOWER
I20260812 00:34:57.644706 24590 raft_consensus.cc:348] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.645176 24590 raft_consensus.cc:374] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:57.645334 24590 raft_consensus.cc:729] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a54d44392da41608e7fcb22095564fe, State: Initialized, Role: FOLLOWER
I20260812 00:34:57.645557 24589 consensus_queue.cc:260] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.645880 24590 consensus_queue.cc:260] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.651659 24589 ts_tablet_manager.cc:1428] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent starting tablet: real 0.020s user 0.016s sys 0.001s
I20260812 00:34:57.654317 24590 ts_tablet_manager.cc:1428] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: Time spent starting tablet: real 0.018s user 0.013s sys 0.005s
W20260812 00:34:57.656634 24268 tablet.cc:2369] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:57.656288 24591 raft_consensus.cc:348] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.656853 24591 raft_consensus.cc:374] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:57.656986 24591 raft_consensus.cc:729] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e715bfb5c8848df92d3ce23d8c1d675, State: Initialized, Role: FOLLOWER
I20260812 00:34:57.657676 24591 consensus_queue.cc:260] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.659365 24591 ts_tablet_manager.cc:1428] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260812 00:34:57.712886 24595 raft_consensus.cc:482] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:57.713269 24595 raft_consensus.cc:504] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.715202 24595 leader_election.cc:288] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241)
I20260812 00:34:57.722986 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.715627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:34751 (local address 127.7.244.65:38439)
0812 00:34:57.715835 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:57.715842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:57.716360 (+ 518us) server_negotiation.cc:407] Connection header received
0812 00:34:57.716557 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:57.716562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:57.716662 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:57.716762 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:57.717487 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.718440 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.720732 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.721856 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.721994 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.722069 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:57.722602 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 00:34:57.722732 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 00:34:57.722995 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.716498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:33907 (local address 127.7.244.67:39241)
0812 00:34:57.716695 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:57.716701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:57.717101 (+ 400us) server_negotiation.cc:407] Connection header received
0812 00:34:57.717235 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:57.717240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:57.717363 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:57.717446 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:57.718099 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.719093 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.720950 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.721956 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.722137 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.722233 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:57.722581 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 00:34:57.722739 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 00:34:57.723119 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.716360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:39241 (local address 127.7.244.66:33907)
0812 00:34:57.716990 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:57.717008 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:57.717125 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:57.717474 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:57.717478 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:57.717503 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:57.717869 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.717880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.719273 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.719277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.720779 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.720794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.722210 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.722215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.722393 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.722436 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 00:34:57.722518 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 00:34:57.722658 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":39,"thread_start_us":350,"threads_started":1}
I20260812 00:34:57.723218 24600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.715554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38439 (local address 127.7.244.66:34751)
0812 00:34:57.716237 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:57.716260 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:57.716411 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:57.716915 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:57.716918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:57.716941 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:57.717326 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.717339 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.718601 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.718609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.720498 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:57.720514 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.722167 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.722174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:57.722426 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.722452 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 00:34:57.722554 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 00:34:57.722723 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":41,"thread_start_us":373,"threads_started":1}
I20260812 00:34:57.724268 24437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c79fc1e5b841b6a465485bb61f2ddd" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a54d44392da41608e7fcb22095564fe" is_pre_election: true
I20260812 00:34:57.724310 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c79fc1e5b841b6a465485bb61f2ddd" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" is_pre_election: true
I20260812 00:34:57.724831 23882 raft_consensus.cc:2454] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 0.
I20260812 00:34:57.724833 24437 raft_consensus.cc:2454] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 0.
I20260812 00:34:57.725540 24038 leader_election.cc:302] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506d7dfe91af4e75bdde8c8a73fedd11, 6a54d44392da41608e7fcb22095564fe; no voters:
I20260812 00:34:57.725978 24595 raft_consensus.cc:2781] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:57.726152 24595 raft_consensus.cc:482] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:57.726279 24595 raft_consensus.cc:3037] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:57.728745 24595 raft_consensus.cc:504] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.729542 24595 leader_election.cc:288] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 election: Requested vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241)
I20260812 00:34:57.730203 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c79fc1e5b841b6a465485bb61f2ddd" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675"
I20260812 00:34:57.730245 24437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2c79fc1e5b841b6a465485bb61f2ddd" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a54d44392da41608e7fcb22095564fe"
I20260812 00:34:57.730434 23882 raft_consensus.cc:3037] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:57.730515 24437 raft_consensus.cc:3037] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:57.732762 23882 raft_consensus.cc:2454] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 1.
I20260812 00:34:57.732887 24437 raft_consensus.cc:2454] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 1.
I20260812 00:34:57.733253 24038 leader_election.cc:302] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506d7dfe91af4e75bdde8c8a73fedd11, 7e715bfb5c8848df92d3ce23d8c1d675; no voters:
I20260812 00:34:57.733642 24595 raft_consensus.cc:2781] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:57.734135 24595 raft_consensus.cc:686] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 LEADER]: Becoming Leader. State: Replica: 506d7dfe91af4e75bdde8c8a73fedd11, State: Running, Role: LEADER
I20260812 00:34:57.734714 24595 consensus_queue.cc:237] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } }
I20260812 00:34:57.739724 23555 catalog_manager.cc:5495] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66). New cstate: current_term: 1 leader_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } health_report { overall_health: HEALTHY } } }
I20260812 00:34:57.758795 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:57.752930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52550 (local address 127.7.244.66:41907)
0812 00:34:57.753132 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:57.753139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:57.753488 (+ 349us) server_negotiation.cc:407] Connection header received
0812 00:34:57.753629 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:57.753634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:57.753767 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:57.753944 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:57.754453 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.755299 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.756748 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:57.757178 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:57.757338 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:57.757728 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:57.758335 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:57.758463 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:57.758502 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 00:34:57.758581 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 00:34:57.767675 24437 raft_consensus.cc:1261] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Refusing update from remote peer 506d7dfe91af4e75bdde8c8a73fedd11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:57.767689 23882 raft_consensus.cc:1261] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Refusing update from remote peer 506d7dfe91af4e75bdde8c8a73fedd11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:57.768769 24604 consensus_queue.cc:1035] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:57.769184 24595 consensus_queue.cc:1035] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:57.782439 24608 mvcc.cc:205] Tried to move back new op lower bound from 7317483101245362176 to 7317483101125185536. Current Snapshot: MvccSnapshot[applied={T|T < 7317483101245362176}]
I20260812 00:34:57.798820 23556 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:57.799170 23556 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:57.803638 23556 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: Sending DeleteTablet for 3 replicas of tablet c2c79fc1e5b841b6a465485bb61f2ddd
I20260812 00:34:57.805083 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet c2c79fc1e5b841b6a465485bb61f2ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:57 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55430
I20260812 00:34:57.805083 24407 tablet_service.cc:1514] Processing DeleteTablet for tablet c2c79fc1e5b841b6a465485bb61f2ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:57 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:43508
I20260812 00:34:57.805546 24128 tablet_service.cc:1514] Processing DeleteTablet for tablet c2c79fc1e5b841b6a465485bb61f2ddd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:57 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:52534
I20260812 00:34:57.809233 23556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:57.812670 23556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:57.817626 24626 tablet_replica.cc:330] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: stopping tablet replica
I20260812 00:34:57.818260 24626 raft_consensus.cc:2229] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:57.818354 24625 tablet_replica.cc:330] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: stopping tablet replica
I20260812 00:34:57.819113 24626 raft_consensus.cc:2258] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:57.819090 24625 raft_consensus.cc:2229] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:57.820048 24625 raft_consensus.cc:2258] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:57.820539 24624 tablet_replica.cc:330] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: stopping tablet replica
I20260812 00:34:57.822943 24625 ts_tablet_manager.cc:1905] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:57.823009 24626 ts_tablet_manager.cc:1905] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:57.828114 23843 tablet_service.cc:1467] Processing CreateTablet for tablet c986af52e5d14019872f7c36f5831884 (DEFAULT_TABLE table=test-table [id=e401dc48115b44009ad47ac5948f4a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:57.828634 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c986af52e5d14019872f7c36f5831884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:57.829901 24128 tablet_service.cc:1467] Processing CreateTablet for tablet c986af52e5d14019872f7c36f5831884 (DEFAULT_TABLE table=test-table [id=e401dc48115b44009ad47ac5948f4a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:57.829954 24624 raft_consensus.cc:2229] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:57.830188 24407 tablet_service.cc:1467] Processing CreateTablet for tablet c986af52e5d14019872f7c36f5831884 (DEFAULT_TABLE table=test-table [id=e401dc48115b44009ad47ac5948f4a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:57.830413 24128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c986af52e5d14019872f7c36f5831884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:57.830582 24624 raft_consensus.cc:2258] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:57.830665 24407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c986af52e5d14019872f7c36f5831884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:57.836190 24625 ts_tablet_manager.cc:1918] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:57.836485 24625 log.cc:1196] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/c2c79fc1e5b841b6a465485bb61f2ddd
I20260812 00:34:57.837059 24625 ts_tablet_manager.cc:1939] T c2c79fc1e5b841b6a465485bb61f2ddd P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting consensus metadata
I20260812 00:34:57.837502 24590 tablet_bootstrap.cc:492] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: Bootstrap starting.
I20260812 00:34:57.837811 24591 tablet_bootstrap.cc:492] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: Bootstrap starting.
I20260812 00:34:57.839316 24590 tablet_bootstrap.cc:654] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:57.839986 24624 ts_tablet_manager.cc:1905] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:57.845512 24626 ts_tablet_manager.cc:1918] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:57.845824 24626 log.cc:1196] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/c2c79fc1e5b841b6a465485bb61f2ddd
I20260812 00:34:57.845975 24591 tablet_bootstrap.cc:654] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:57.846493 24626 ts_tablet_manager.cc:1939] T c2c79fc1e5b841b6a465485bb61f2ddd P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting consensus metadata
I20260812 00:34:57.849728 24624 ts_tablet_manager.cc:1918] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:57.850013 24624 log.cc:1196] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/c2c79fc1e5b841b6a465485bb61f2ddd
I20260812 00:34:57.850801 24624 ts_tablet_manager.cc:1939] T c2c79fc1e5b841b6a465485bb61f2ddd P 6a54d44392da41608e7fcb22095564fe: Deleting consensus metadata
I20260812 00:34:57.851390 24590 tablet_bootstrap.cc:492] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: No bootstrap required, opened a new log
I20260812 00:34:57.851564 24590 ts_tablet_manager.cc:1397] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.001s
I20260812 00:34:57.852491 24590 raft_consensus.cc:348] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:57.852736 24590 raft_consensus.cc:374] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:57.852841 24590 raft_consensus.cc:729] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a54d44392da41608e7fcb22095564fe, State: Initialized, Role: FOLLOWER
I20260812 00:34:57.853461 23522 catalog_manager.cc:4841] TS 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907): tablet c2c79fc1e5b841b6a465485bb61f2ddd (table test-table [id=ef9e9c8bd13b4b059829dfad8c659de2]) successfully deleted
I20260812 00:34:57.853204 24590 consensus_queue.cc:260] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:57.853883 23523 catalog_manager.cc:4841] TS 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439): tablet c2c79fc1e5b841b6a465485bb61f2ddd (table test-table [id=ef9e9c8bd13b4b059829dfad8c659de2]) successfully deleted
I20260812 00:34:57.854336 24589 tablet_bootstrap.cc:492] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: Bootstrap starting.
I20260812 00:34:57.855057 24591 tablet_bootstrap.cc:492] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: No bootstrap required, opened a new log
I20260812 00:34:57.855224 24591 ts_tablet_manager.cc:1397] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260812 00:34:57.856104 24589 tablet_bootstrap.cc:654] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:57.856122 24591 raft_consensus.cc:348] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:57.856369 24591 raft_consensus.cc:374] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:57.856487 24591 raft_consensus.cc:729] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e715bfb5c8848df92d3ce23d8c1d675, State: Initialized, Role: FOLLOWER
I20260812 00:34:57.856662 24590 ts_tablet_manager.cc:1428] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260812 00:34:57.856840 24591 consensus_queue.cc:260] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:57.858888 24591 ts_tablet_manager.cc:1428] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260812 00:34:57.860355 23523 catalog_manager.cc:4841] TS 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241): tablet c2c79fc1e5b841b6a465485bb61f2ddd (table test-table [id=ef9e9c8bd13b4b059829dfad8c659de2]) successfully deleted
I20260812 00:34:57.862064 24589 tablet_bootstrap.cc:492] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: No bootstrap required, opened a new log
I20260812 00:34:57.862260 24589 ts_tablet_manager.cc:1397] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 00:34:57.863164 24589 raft_consensus.cc:348] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:57.863407 24589 raft_consensus.cc:374] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:57.863514 24589 raft_consensus.cc:729] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506d7dfe91af4e75bdde8c8a73fedd11, State: Initialized, Role: FOLLOWER
I20260812 00:34:57.863809 24589 consensus_queue.cc:260] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:57.864522 24589 ts_tablet_manager.cc:1428] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260812 00:34:57.882995 24542 tablet.cc:2369] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:34:58.004953 24596 raft_consensus.cc:482] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:58.005316 24596 raft_consensus.cc:504] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.007481 24596 leader_election.cc:288] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907)
I20260812 00:34:58.016620 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.008014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:55757 (local address 127.7.244.65:38439)
0812 00:34:58.008218 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.008224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.009574 (+ 1350us) server_negotiation.cc:407] Connection header received
0812 00:34:58.009924 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.009929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.010004 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.010096 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:58.010806 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.011935 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.014385 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.015307 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.015518 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.015615 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:58.016152 (+ 537us) server_negotiation.cc:299] Negotiation successful
0812 00:34:58.016313 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:34:58.016752 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.007856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38439 (local address 127.7.244.67:55757)
0812 00:34:58.009427 (+ 1571us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:58.009447 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:58.009692 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:58.010138 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:58.010143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:58.010180 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:58.010622 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.010637 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.012093 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.012099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.014204 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.014216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.015592 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.015599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.015913 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.015974 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 00:34:58.016099 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 00:34:58.016269 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1403,"mutex_wait_us":55,"thread_start_us":637,"threads_started":1}
I20260812 00:34:58.016808 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.008235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:37199 (local address 127.7.244.66:41907)
0812 00:34:58.008447 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.008453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.008956 (+ 503us) server_negotiation.cc:407] Connection header received
0812 00:34:58.009376 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.009383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.009476 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.009595 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:34:58.010316 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.011329 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.014076 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.015406 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.015599 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.015686 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:58.016288 (+ 602us) server_negotiation.cc:299] Negotiation successful
0812 00:34:58.016452 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 00:34:58.017573 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.007791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:41907 (local address 127.7.244.67:37199)
0812 00:34:58.008826 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:58.008848 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:58.009102 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:58.009660 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:58.009665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:58.009696 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:34:58.010064 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.010077 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.011496 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.011502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.013836 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.013857 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.015812 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.015818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.016067 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.016100 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 00:34:58.017000 (+ 900us) client_negotiation.cc:240] Negotiation successful
0812 00:34:58.017201 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":64,"thread_start_us":1097,"threads_started":1}
I20260812 00:34:58.017890 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c986af52e5d14019872f7c36f5831884" candidate_uuid: "6a54d44392da41608e7fcb22095564fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" is_pre_election: true
I20260812 00:34:58.018175 23882 raft_consensus.cc:2454] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a54d44392da41608e7fcb22095564fe in term 0.
I20260812 00:34:58.018766 24310 leader_election.cc:302] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a54d44392da41608e7fcb22095564fe, 7e715bfb5c8848df92d3ce23d8c1d675; no voters:
I20260812 00:34:58.019223 24596 raft_consensus.cc:2781] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:58.019073 24164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c986af52e5d14019872f7c36f5831884" candidate_uuid: "6a54d44392da41608e7fcb22095564fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" is_pre_election: true
I20260812 00:34:58.019361 24596 raft_consensus.cc:482] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:58.019479 24596 raft_consensus.cc:3037] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.019552 24164 raft_consensus.cc:2454] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a54d44392da41608e7fcb22095564fe in term 0.
I20260812 00:34:58.022415 24596 raft_consensus.cc:504] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.023262 24596 leader_election.cc:288] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [CANDIDATE]: Term 1 election: Requested vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907)
I20260812 00:34:58.023864 24164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c986af52e5d14019872f7c36f5831884" candidate_uuid: "6a54d44392da41608e7fcb22095564fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506d7dfe91af4e75bdde8c8a73fedd11"
I20260812 00:34:58.023904 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c986af52e5d14019872f7c36f5831884" candidate_uuid: "6a54d44392da41608e7fcb22095564fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675"
I20260812 00:34:58.024120 24164 raft_consensus.cc:3037] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.024163 23882 raft_consensus.cc:3037] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.026861 23882 raft_consensus.cc:2454] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a54d44392da41608e7fcb22095564fe in term 1.
I20260812 00:34:58.027426 24164 raft_consensus.cc:2454] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a54d44392da41608e7fcb22095564fe in term 1.
I20260812 00:34:58.027443 24310 leader_election.cc:302] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a54d44392da41608e7fcb22095564fe, 7e715bfb5c8848df92d3ce23d8c1d675; no voters:
I20260812 00:34:58.027807 24596 raft_consensus.cc:2781] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:58.028510 24596 raft_consensus.cc:686] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 1 LEADER]: Becoming Leader. State: Replica: 6a54d44392da41608e7fcb22095564fe, State: Running, Role: LEADER
I20260812 00:34:58.029047 24596 consensus_queue.cc:237] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.033175 23555 catalog_manager.cc:5495] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a54d44392da41608e7fcb22095564fe (127.7.244.67). New cstate: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } health_report { overall_health: HEALTHY } } }
W20260812 00:34:58.086593 23988 tablet.cc:2369] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 00:34:58 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494894, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.126@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:58.120076 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.104055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33135 (local address 127.0.0.1:60198)
0812 00:34:58.104601 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:58.104626 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:58.104797 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:58.105230 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:58.105239 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:58.106232 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:34:58.106538 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.106555 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.107683 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.107689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.109231 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.109252 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.110133 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.110139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.110328 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.111180 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:34:58.111206 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:34:58.114090 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:34:58.117334 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:58.117353 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:58.117363 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:58.117853 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:58.118243 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:34:58.118248 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:34:58.118252 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:34:58.118403 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:34:58.118977 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:34:58.118987 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:34:58.119165 (+ 178us) client_negotiation.cc:769] Sending connection context
0812 00:34:58.119333 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 00:34:58.119645 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":262,"threads_started":1}
I20260812 00:34:58.120162 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.104144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.7.244.126:33135)
0812 00:34:58.104924 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.104928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.104947 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:34:58.104995 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.104999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.105059 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.105147 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:58.106750 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.107515 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.109429 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.109939 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.110044 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.114343 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:58.114349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:58.114352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:58.114388 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:58.117140 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:58.117993 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:58.117997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:58.118000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:58.118132 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:58.118515 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:58.118518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:58.118520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:58.118839 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:58.118976 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:58.119445 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 00:34:58.119789 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":679,"thread_start_us":569,"threads_started":1}
I20260812 00:34:58.127117 23555 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:60198:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 00:34:58.128547 23555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:58.137353 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa1872a349e4e2899adbe87b3da5efa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b14c684b8ff04c7485f42e66776bff11]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:34:58.137353 24128 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa1872a349e4e2899adbe87b3da5efa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b14c684b8ff04c7485f42e66776bff11]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:34:58.137686 24407 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa1872a349e4e2899adbe87b3da5efa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b14c684b8ff04c7485f42e66776bff11]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 00:34:58.138015 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa1872a349e4e2899adbe87b3da5efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.138031 24128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa1872a349e4e2899adbe87b3da5efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.138259 24407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa1872a349e4e2899adbe87b3da5efa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.145848 24591 tablet_bootstrap.cc:492] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: Bootstrap starting.
I20260812 00:34:58.146034 24590 tablet_bootstrap.cc:492] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: Bootstrap starting.
I20260812 00:34:58.147302 24591 tablet_bootstrap.cc:654] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.147948 24590 tablet_bootstrap.cc:654] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.148480 24589 tablet_bootstrap.cc:492] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: Bootstrap starting.
I20260812 00:34:58.150333 24589 tablet_bootstrap.cc:654] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.150799 24590 tablet_bootstrap.cc:492] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: No bootstrap required, opened a new log
I20260812 00:34:58.151057 24590 ts_tablet_manager.cc:1397] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260812 00:34:58.151943 24591 tablet_bootstrap.cc:492] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: No bootstrap required, opened a new log
I20260812 00:34:58.151830 24590 raft_consensus.cc:348] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.152123 24591 ts_tablet_manager.cc:1397] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260812 00:34:58.152130 24590 raft_consensus.cc:374] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.152217 24590 raft_consensus.cc:729] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a54d44392da41608e7fcb22095564fe, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.152848 24591 raft_consensus.cc:348] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.153106 24591 raft_consensus.cc:374] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.152966 24590 consensus_queue.cc:260] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.153191 24591 raft_consensus.cc:729] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e715bfb5c8848df92d3ce23d8c1d675, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.153491 24591 consensus_queue.cc:260] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.153460 24596 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.153766 24596 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:58.153899 24597 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.154246 24597 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:58.154886 24590 ts_tablet_manager.cc:1428] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260812 00:34:58.156024 24591 ts_tablet_manager.cc:1428] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 00:34:58.162611 24589 tablet_bootstrap.cc:492] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: No bootstrap required, opened a new log
I20260812 00:34:58.162827 24589 ts_tablet_manager.cc:1397] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.003s
I20260812 00:34:58.163589 24589 raft_consensus.cc:348] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.163865 24589 raft_consensus.cc:374] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.163946 24589 raft_consensus.cc:729] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506d7dfe91af4e75bdde8c8a73fedd11, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.164191 24589 consensus_queue.cc:260] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.164829 24589 ts_tablet_manager.cc:1428] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 00:34:58.164620 24617 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.165055 24617 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:58.193571 24596 raft_consensus.cc:482] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:58.193836 24596 raft_consensus.cc:504] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.194633 24596 leader_election.cc:288] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907)
I20260812 00:34:58.195242 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa1872a349e4e2899adbe87b3da5efa" candidate_uuid: "6a54d44392da41608e7fcb22095564fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" is_pre_election: true
I20260812 00:34:58.195434 24164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa1872a349e4e2899adbe87b3da5efa" candidate_uuid: "6a54d44392da41608e7fcb22095564fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" is_pre_election: true
I20260812 00:34:58.195580 23882 raft_consensus.cc:2454] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a54d44392da41608e7fcb22095564fe in term 0.
I20260812 00:34:58.195725 24164 raft_consensus.cc:2454] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a54d44392da41608e7fcb22095564fe in term 0.
I20260812 00:34:58.196115 24310 leader_election.cc:302] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a54d44392da41608e7fcb22095564fe, 7e715bfb5c8848df92d3ce23d8c1d675; no voters:
I20260812 00:34:58.196426 24596 raft_consensus.cc:2781] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:58.196528 24596 raft_consensus.cc:482] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:58.196646 24596 raft_consensus.cc:3037] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.198832 24596 raft_consensus.cc:504] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.199506 24596 leader_election.cc:288] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [CANDIDATE]: Term 1 election: Requested vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907)
I20260812 00:34:58.200105 24164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa1872a349e4e2899adbe87b3da5efa" candidate_uuid: "6a54d44392da41608e7fcb22095564fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506d7dfe91af4e75bdde8c8a73fedd11"
I20260812 00:34:58.200105 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa1872a349e4e2899adbe87b3da5efa" candidate_uuid: "6a54d44392da41608e7fcb22095564fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675"
I20260812 00:34:58.200344 24164 raft_consensus.cc:3037] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.200378 23882 raft_consensus.cc:3037] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.203243 24164 raft_consensus.cc:2454] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a54d44392da41608e7fcb22095564fe in term 1.
I20260812 00:34:58.203577 23882 raft_consensus.cc:2454] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a54d44392da41608e7fcb22095564fe in term 1.
I20260812 00:34:58.203785 24308 leader_election.cc:302] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506d7dfe91af4e75bdde8c8a73fedd11, 6a54d44392da41608e7fcb22095564fe; no voters:
I20260812 00:34:58.204254 24596 raft_consensus.cc:2781] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:58.204412 24596 raft_consensus.cc:686] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [term 1 LEADER]: Becoming Leader. State: Replica: 6a54d44392da41608e7fcb22095564fe, State: Running, Role: LEADER
I20260812 00:34:58.204663 24596 consensus_queue.cc:237] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.207259 24634 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: TxnStatusTablet state changed. Reason: New leader 6a54d44392da41608e7fcb22095564fe. Latest consensus state: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.207670 24634 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:34:58.208168 23556 catalog_manager.cc:5495] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a54d44392da41608e7fcb22095564fe (127.7.244.67). New cstate: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } health_report { overall_health: HEALTHY } } }
I20260812 00:34:58.209808 24667 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 00:34:58.210237 24667 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 00:34:58.238996 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.230207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43512 (local address 127.7.244.67:39241)
0812 00:34:58.231428 (+ 1221us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.231434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.231453 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:34:58.231560 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.231565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.231677 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.231834 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:58.232326 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.233614 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.235741 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.236454 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.236589 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.237237 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:58.238121 (+ 884us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:58.238279 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:58.238493 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 00:34:58.238612 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1075,"thread_start_us":921,"threads_started":1}
I20260812 00:34:58.239002 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.230202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:39241 (local address 127.0.0.1:43512)
0812 00:34:58.230650 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:58.230674 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:58.230835 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:58.231914 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:58.231920 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:58.231936 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:34:58.232147 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.232159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.233793 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.233799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.235559 (+ 1760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.235582 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.236663 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.236671 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.236908 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.237005 (+ 97us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:34:58.238297 (+ 1292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:34:58.238334 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 00:34:58.238446 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 00:34:58.238646 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253}
I20260812 00:34:58.244556 24164 raft_consensus.cc:1261] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Refusing update from remote peer 6a54d44392da41608e7fcb22095564fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:58.244866 23882 raft_consensus.cc:1261] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Refusing update from remote peer 6a54d44392da41608e7fcb22095564fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:58.245621 24596 consensus_queue.cc:1035] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:58.246421 24596 consensus_queue.cc:1035] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:58.260622 24597 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: TxnStatusTablet state changed. Reason: New leader 6a54d44392da41608e7fcb22095564fe. Latest consensus state: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.260973 24597 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:58.263818 24596 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.264151 24596 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:34:58.264382 24611 mvcc.cc:205] Tried to move back new op lower bound from 7317483103199944704 to 7317483103049068544. Current Snapshot: MvccSnapshot[applied={T|T < 7317483103199944704}]
I20260812 00:34:58.266716 24617 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: TxnStatusTablet state changed. Reason: New leader 6a54d44392da41608e7fcb22095564fe. Latest consensus state: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.266997 24617 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:58.269313 24609 mvcc.cc:205] Tried to move back new op lower bound from 7317483103199944704 to 7317483103049068544. Current Snapshot: MvccSnapshot[applied={T|T < 7317483103199944704}]
I20260812 00:34:58.269695 24597 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.269958 24597 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 7e715bfb5c8848df92d3ce23d8c1d675: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:58.272119 24634 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.272377 24634 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 6a54d44392da41608e7fcb22095564fe: This TxnStatusTablet replica's current role is: LEADER
I20260812 00:34:58.279285 24617 tablet_replica.cc:439] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a54d44392da41608e7fcb22095564fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } } }
I20260812 00:34:58.279598 24617 tablet_replica.cc:442] T 8fa1872a349e4e2899adbe87b3da5efa P 506d7dfe91af4e75bdde8c8a73fedd11: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 00:34:58.286695 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.279048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.7.244.67:39241)
0812 00:34:58.279457 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.279463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.280017 (+ 554us) server_negotiation.cc:407] Connection header received
0812 00:34:58.280326 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.280331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.280431 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.280539 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:58.281115 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.282349 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.284363 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.285062 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.285192 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.285857 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:58.286008 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:58.286105 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:58.286262 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 00:34:58.286354 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260812 00:34:58.304839 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.299046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:40813 (local address 127.7.244.67:39241)
0812 00:34:58.299210 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.299215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.299823 (+ 608us) server_negotiation.cc:407] Connection header received
0812 00:34:58.300043 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.300047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.300121 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.300193 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:58.300718 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.301546 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.303065 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.303539 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.303653 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.304102 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:58.304213 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:58.304286 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:58.304415 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 00:34:58.304490 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:34:58.304919 24685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.298994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:39241 (local address 127.7.244.67:40813)
0812 00:34:58.299717 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:58.299733 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:58.299897 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:58.300219 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:58.300224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:58.300233 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:34:58.300561 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.300570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.301674 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.301683 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.302922 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:58.302937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.303733 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.303739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:58.303858 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.303919 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:34:58.304306 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:34:58.304318 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 00:34:58.304387 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 00:34:58.304545 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":406,"threads_started":1}
I20260812 00:34:58.316267 24164 raft_consensus.cc:1261] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Refusing update from remote peer 6a54d44392da41608e7fcb22095564fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:58.316411 23880 raft_consensus.cc:1261] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Refusing update from remote peer 6a54d44392da41608e7fcb22095564fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:58.316911 24596 consensus_queue.cc:1035] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:58.317219 24689 consensus_queue.cc:1035] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:58.394690 23555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:58.394999 23555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:58.399156 23555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: Sending DeleteTablet for 3 replicas of tablet c986af52e5d14019872f7c36f5831884
I20260812 00:34:58.400339 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet c986af52e5d14019872f7c36f5831884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:58 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55430
I20260812 00:34:58.401275 24128 tablet_service.cc:1514] Processing DeleteTablet for tablet c986af52e5d14019872f7c36f5831884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:58 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:52534
I20260812 00:34:58.401579 24698 tablet_replica.cc:330] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: stopping tablet replica
I20260812 00:34:58.401805 24698 raft_consensus.cc:2229] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:58.402099 24698 raft_consensus.cc:2258] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:58.402446 24407 tablet_service.cc:1514] Processing DeleteTablet for tablet c986af52e5d14019872f7c36f5831884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:58 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:43508
I20260812 00:34:58.403199 24698 ts_tablet_manager.cc:1905] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:58.410929 24702 tablet_replica.cc:330] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: stopping tablet replica
I20260812 00:34:58.411207 24702 raft_consensus.cc:2229] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:58.411877 24702 raft_consensus.cc:2258] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:58.412115 24698 ts_tablet_manager.cc:1918] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:58.412284 24698 log.cc:1196] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/c986af52e5d14019872f7c36f5831884
I20260812 00:34:58.412779 24698 ts_tablet_manager.cc:1939] T c986af52e5d14019872f7c36f5831884 P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting consensus metadata
I20260812 00:34:58.413164 24702 ts_tablet_manager.cc:1905] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:58.414076 23523 catalog_manager.cc:4841] TS 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439): tablet c986af52e5d14019872f7c36f5831884 (table test-table [id=e401dc48115b44009ad47ac5948f4a63]) successfully deleted
I20260812 00:34:58.419950 24702 ts_tablet_manager.cc:1918] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:58.420147 24702 log.cc:1196] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/c986af52e5d14019872f7c36f5831884
I20260812 00:34:58.420625 24702 ts_tablet_manager.cc:1939] T c986af52e5d14019872f7c36f5831884 P 6a54d44392da41608e7fcb22095564fe: Deleting consensus metadata
I20260812 00:34:58.421370 24700 tablet_replica.cc:330] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: stopping tablet replica
I20260812 00:34:58.421588 24700 raft_consensus.cc:2229] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:58.421854 24700 raft_consensus.cc:2258] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:58.422470 23523 catalog_manager.cc:4841] TS 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241): tablet c986af52e5d14019872f7c36f5831884 (table test-table [id=e401dc48115b44009ad47ac5948f4a63]) successfully deleted
I20260812 00:34:58.423533 24700 ts_tablet_manager.cc:1905] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:58.430434 24700 ts_tablet_manager.cc:1918] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:58.430601 24700 log.cc:1196] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/c986af52e5d14019872f7c36f5831884
I20260812 00:34:58.431103 24700 ts_tablet_manager.cc:1939] T c986af52e5d14019872f7c36f5831884 P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting consensus metadata
I20260812 00:34:58.432338 23522 catalog_manager.cc:4841] TS 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907): tablet c986af52e5d14019872f7c36f5831884 (table test-table [id=e401dc48115b44009ad47ac5948f4a63]) successfully deleted
Aug 12 00:34:58 dist-test-slave-gjw7 krb5kdc[23466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 00:34:58 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:34:58 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 00:34:58.485493 24653 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:34:58.473339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60204 (local address 127.7.244.126:33135)
0812 00:34:58.473572 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.473579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.473965 (+ 386us) server_negotiation.cc:407] Connection header received
0812 00:34:58.474162 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.474168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.474258 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.474412 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:58.475828 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.476919 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.478484 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.479141 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.479274 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.485174 (+ 5900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60204: BlockingRecv error: recv got EOF from 127.0.0.1:60204 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 00:34:58 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM
I20260812 00:34:58.504799 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.488229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.7.244.126:33135)
0812 00:34:58.488472 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.488478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.488628 (+ 150us) server_negotiation.cc:407] Connection header received
0812 00:34:58.488816 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.488822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.488892 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.488981 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:58.490371 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.491107 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.492387 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.492820 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.492935 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.497352 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:58.497360 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:58.497365 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:58.497426 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:58.500899 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:58.501884 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:58.501891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:58.501895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:58.502130 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:58.502786 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:58.502794 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:58.502805 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:58.503319 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:58.503690 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:58.504043 (+ 353us) server_negotiation.cc:299] Negotiation successful
0812 00:34:58.504373 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260812 00:34:58.509426 23555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:34:58.510386 23555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:58.519296 24128 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5d67fda9a141d9bb8683c3346d97b7 (DEFAULT_TABLE table=test-table [id=3cdcbf20f2ca448291399f74fdc92127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:58.519481 24407 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5d67fda9a141d9bb8683c3346d97b7 (DEFAULT_TABLE table=test-table [id=3cdcbf20f2ca448291399f74fdc92127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:58.519508 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 4e5d67fda9a141d9bb8683c3346d97b7 (DEFAULT_TABLE table=test-table [id=3cdcbf20f2ca448291399f74fdc92127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:58.519970 24128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5d67fda9a141d9bb8683c3346d97b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.519975 24407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5d67fda9a141d9bb8683c3346d97b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.520030 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e5d67fda9a141d9bb8683c3346d97b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.527462 24591 tablet_bootstrap.cc:492] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: Bootstrap starting.
I20260812 00:34:58.527588 24589 tablet_bootstrap.cc:492] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: Bootstrap starting.
I20260812 00:34:58.529222 24589 tablet_bootstrap.cc:654] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.529354 24591 tablet_bootstrap.cc:654] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.529482 24590 tablet_bootstrap.cc:492] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: Bootstrap starting.
I20260812 00:34:58.531330 24590 tablet_bootstrap.cc:654] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.539798 24591 tablet_bootstrap.cc:492] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: No bootstrap required, opened a new log
I20260812 00:34:58.540016 24591 ts_tablet_manager.cc:1397] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260812 00:34:58.540207 24589 tablet_bootstrap.cc:492] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: No bootstrap required, opened a new log
I20260812 00:34:58.540241 24590 tablet_bootstrap.cc:492] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: No bootstrap required, opened a new log
I20260812 00:34:58.540429 24590 ts_tablet_manager.cc:1397] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.001s
I20260812 00:34:58.540570 24589 ts_tablet_manager.cc:1397] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260812 00:34:58.540995 24591 raft_consensus.cc:348] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.541373 24591 raft_consensus.cc:374] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.541478 24591 raft_consensus.cc:729] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e715bfb5c8848df92d3ce23d8c1d675, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.541356 24590 raft_consensus.cc:348] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.541643 24590 raft_consensus.cc:374] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.541751 24590 raft_consensus.cc:729] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a54d44392da41608e7fcb22095564fe, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.541785 24591 consensus_queue.cc:260] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.541996 24590 consensus_queue.cc:260] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.542582 24590 ts_tablet_manager.cc:1428] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 00:34:58.543401 24589 raft_consensus.cc:348] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.543761 24589 raft_consensus.cc:374] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.543915 24589 raft_consensus.cc:729] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506d7dfe91af4e75bdde8c8a73fedd11, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.544195 24589 consensus_queue.cc:260] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.544749 24589 ts_tablet_manager.cc:1428] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260812 00:34:58.545703 24591 ts_tablet_manager.cc:1428] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260812 00:34:58.757459 24617 raft_consensus.cc:482] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:58.757725 24617 raft_consensus.cc:504] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.758622 24617 leader_election.cc:288] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241)
I20260812 00:34:58.759251 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5d67fda9a141d9bb8683c3346d97b7" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" is_pre_election: true
I20260812 00:34:58.759305 24437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5d67fda9a141d9bb8683c3346d97b7" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a54d44392da41608e7fcb22095564fe" is_pre_election: true
I20260812 00:34:58.759567 23882 raft_consensus.cc:2454] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 0.
I20260812 00:34:58.759595 24437 raft_consensus.cc:2454] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 0.
I20260812 00:34:58.760242 24038 leader_election.cc:302] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506d7dfe91af4e75bdde8c8a73fedd11, 7e715bfb5c8848df92d3ce23d8c1d675; no voters:
I20260812 00:34:58.760833 24617 raft_consensus.cc:2781] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:58.760974 24617 raft_consensus.cc:482] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:58.761076 24617 raft_consensus.cc:3037] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.764358 24617 raft_consensus.cc:504] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.765133 24617 leader_election.cc:288] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 election: Requested vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241)
I20260812 00:34:58.765817 23882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5d67fda9a141d9bb8683c3346d97b7" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675"
I20260812 00:34:58.765957 24437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e5d67fda9a141d9bb8683c3346d97b7" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a54d44392da41608e7fcb22095564fe"
I20260812 00:34:58.766103 23882 raft_consensus.cc:3037] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.766211 24437 raft_consensus.cc:3037] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:58.768627 24437 raft_consensus.cc:2454] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 1.
I20260812 00:34:58.769209 23882 raft_consensus.cc:2454] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 1.
I20260812 00:34:58.769249 24038 leader_election.cc:302] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506d7dfe91af4e75bdde8c8a73fedd11, 6a54d44392da41608e7fcb22095564fe; no voters:
I20260812 00:34:58.769817 24617 raft_consensus.cc:2781] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:58.770314 24617 raft_consensus.cc:686] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 LEADER]: Becoming Leader. State: Replica: 506d7dfe91af4e75bdde8c8a73fedd11, State: Running, Role: LEADER
I20260812 00:34:58.770635 24617 consensus_queue.cc:237] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.773856 23546 catalog_manager.cc:5495] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66). New cstate: current_term: 1 leader_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } health_report { overall_health: UNKNOWN } } }
I20260812 00:34:58.839135 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:58.833711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52552 (local address 127.7.244.66:41907)
0812 00:34:58.834388 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:58.834394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:58.834415 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:34:58.834476 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:58.834480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:58.834543 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:58.834629 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:58.835034 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.835854 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.837146 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:58.837717 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:58.837830 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:58.838244 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:58.838366 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:58.838449 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:58.838594 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 00:34:58.838677 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":325,"threads_started":1}
I20260812 00:34:58.842453 24437 raft_consensus.cc:1261] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Refusing update from remote peer 506d7dfe91af4e75bdde8c8a73fedd11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:58.842602 23882 raft_consensus.cc:1261] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Refusing update from remote peer 506d7dfe91af4e75bdde8c8a73fedd11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:58.843171 24729 consensus_queue.cc:1035] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:58.843590 24617 consensus_queue.cc:1035] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:58.871778 23546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:58.872160 23546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:58.875712 23546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: Sending DeleteTablet for 3 replicas of tablet 4e5d67fda9a141d9bb8683c3346d97b7
I20260812 00:34:58.876777 24128 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e5d67fda9a141d9bb8683c3346d97b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:58 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:52534
I20260812 00:34:58.876863 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e5d67fda9a141d9bb8683c3346d97b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:58 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55430
I20260812 00:34:58.877095 24700 tablet_replica.cc:330] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: stopping tablet replica
I20260812 00:34:58.877347 24700 raft_consensus.cc:2229] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:58.877614 24407 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e5d67fda9a141d9bb8683c3346d97b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:58 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:43508
I20260812 00:34:58.877719 24700 raft_consensus.cc:2258] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:58.877974 24702 tablet_replica.cc:330] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: stopping tablet replica
I20260812 00:34:58.878036 24698 tablet_replica.cc:330] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: stopping tablet replica
I20260812 00:34:58.878172 24702 raft_consensus.cc:2229] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:58.878242 24698 raft_consensus.cc:2229] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:58.878432 24702 raft_consensus.cc:2258] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:58.878485 24698 raft_consensus.cc:2258] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:58.879495 23556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 00:34:58.880029 24702 ts_tablet_manager.cc:1905] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:58.880016 24698 ts_tablet_manager.cc:1905] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 00:34:58.880343 23556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:34:58.882324 24700 ts_tablet_manager.cc:1905] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:58.887631 24702 ts_tablet_manager.cc:1918] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:58.887850 24702 log.cc:1196] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/4e5d67fda9a141d9bb8683c3346d97b7
I20260812 00:34:58.888322 24702 ts_tablet_manager.cc:1939] T 4e5d67fda9a141d9bb8683c3346d97b7 P 6a54d44392da41608e7fcb22095564fe: Deleting consensus metadata
I20260812 00:34:58.889847 23523 catalog_manager.cc:4841] TS 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241): tablet 4e5d67fda9a141d9bb8683c3346d97b7 (table test-table [id=3cdcbf20f2ca448291399f74fdc92127]) successfully deleted
I20260812 00:34:58.892063 23843 tablet_service.cc:1467] Processing CreateTablet for tablet 655140effa224573ae7fc7a5b0141bcd (DEFAULT_TABLE table=test-table [id=6e7377ae8ebb46cdb206052f9642f807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:58.892464 24700 ts_tablet_manager.cc:1918] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:58.892583 24700 log.cc:1196] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/4e5d67fda9a141d9bb8683c3346d97b7
I20260812 00:34:58.892589 23843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655140effa224573ae7fc7a5b0141bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.892892 24700 ts_tablet_manager.cc:1939] T 4e5d67fda9a141d9bb8683c3346d97b7 P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting consensus metadata
I20260812 00:34:58.894099 24407 tablet_service.cc:1467] Processing CreateTablet for tablet 655140effa224573ae7fc7a5b0141bcd (DEFAULT_TABLE table=test-table [id=6e7377ae8ebb46cdb206052f9642f807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:58.894572 24407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655140effa224573ae7fc7a5b0141bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.895565 24698 ts_tablet_manager.cc:1918] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:34:58.895716 24698 log.cc:1196] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/4e5d67fda9a141d9bb8683c3346d97b7
I20260812 00:34:58.896167 24698 ts_tablet_manager.cc:1939] T 4e5d67fda9a141d9bb8683c3346d97b7 P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting consensus metadata
I20260812 00:34:58.897524 23523 catalog_manager.cc:4841] TS 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439): tablet 4e5d67fda9a141d9bb8683c3346d97b7 (table test-table [id=3cdcbf20f2ca448291399f74fdc92127]) successfully deleted
I20260812 00:34:58.900125 24590 tablet_bootstrap.cc:492] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: Bootstrap starting.
I20260812 00:34:58.900231 24591 tablet_bootstrap.cc:492] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: Bootstrap starting.
I20260812 00:34:58.900790 23522 catalog_manager.cc:4841] TS 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907): tablet 4e5d67fda9a141d9bb8683c3346d97b7 (table test-table [id=3cdcbf20f2ca448291399f74fdc92127]) successfully deleted
I20260812 00:34:58.902010 24128 tablet_service.cc:1467] Processing CreateTablet for tablet 655140effa224573ae7fc7a5b0141bcd (DEFAULT_TABLE table=test-table [id=6e7377ae8ebb46cdb206052f9642f807]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:34:58.902128 24591 tablet_bootstrap.cc:654] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.902521 24128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 655140effa224573ae7fc7a5b0141bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:34:58.904614 24590 tablet_bootstrap.cc:654] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.909642 24589 tablet_bootstrap.cc:492] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: Bootstrap starting.
I20260812 00:34:58.911362 24589 tablet_bootstrap.cc:654] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: Neither blocks nor log segments found. Creating new log.
I20260812 00:34:58.912307 24590 tablet_bootstrap.cc:492] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: No bootstrap required, opened a new log
I20260812 00:34:58.912556 24590 ts_tablet_manager.cc:1397] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260812 00:34:58.913018 24591 tablet_bootstrap.cc:492] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: No bootstrap required, opened a new log
I20260812 00:34:58.913251 24591 ts_tablet_manager.cc:1397] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260812 00:34:58.913712 24590 raft_consensus.cc:348] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.914062 24590 raft_consensus.cc:374] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.914175 24590 raft_consensus.cc:729] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a54d44392da41608e7fcb22095564fe, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.914206 24591 raft_consensus.cc:348] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.914513 24591 raft_consensus.cc:374] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.914606 24591 raft_consensus.cc:729] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e715bfb5c8848df92d3ce23d8c1d675, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.914603 24590 consensus_queue.cc:260] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.914932 24591 consensus_queue.cc:260] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.915314 24590 ts_tablet_manager.cc:1428] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 00:34:58.916700 24591 ts_tablet_manager.cc:1428] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 00:34:58.923259 24589 tablet_bootstrap.cc:492] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: No bootstrap required, opened a new log
I20260812 00:34:58.923484 24589 ts_tablet_manager.cc:1397] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260812 00:34:58.924220 24589 raft_consensus.cc:348] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.924551 24589 raft_consensus.cc:374] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:34:58.924641 24589 raft_consensus.cc:729] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506d7dfe91af4e75bdde8c8a73fedd11, State: Initialized, Role: FOLLOWER
I20260812 00:34:58.924871 24589 consensus_queue.cc:260] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:58.925390 24589 ts_tablet_manager.cc:1428] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 00:34:59.043119 24617 raft_consensus.cc:482] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:34:59.043398 24617 raft_consensus.cc:504] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:59.044294 24617 leader_election.cc:288] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241)
I20260812 00:34:59.045017 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655140effa224573ae7fc7a5b0141bcd" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" is_pre_election: true
I20260812 00:34:59.045092 24437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655140effa224573ae7fc7a5b0141bcd" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a54d44392da41608e7fcb22095564fe" is_pre_election: true
I20260812 00:34:59.045353 23880 raft_consensus.cc:2454] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 0.
I20260812 00:34:59.045387 24437 raft_consensus.cc:2454] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 0.
I20260812 00:34:59.045915 24038 leader_election.cc:302] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506d7dfe91af4e75bdde8c8a73fedd11, 7e715bfb5c8848df92d3ce23d8c1d675; no voters:
I20260812 00:34:59.046253 24617 raft_consensus.cc:2781] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:34:59.046389 24617 raft_consensus.cc:482] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:34:59.046494 24617 raft_consensus.cc:3037] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:59.049722 24617 raft_consensus.cc:504] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:59.050501 24617 leader_election.cc:288] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 election: Requested vote from peers 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439), 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241)
I20260812 00:34:59.051190 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655140effa224573ae7fc7a5b0141bcd" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e715bfb5c8848df92d3ce23d8c1d675"
I20260812 00:34:59.051301 24437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "655140effa224573ae7fc7a5b0141bcd" candidate_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a54d44392da41608e7fcb22095564fe"
I20260812 00:34:59.051445 23880 raft_consensus.cc:3037] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:59.051545 24437 raft_consensus.cc:3037] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:34:59.053742 23880 raft_consensus.cc:2454] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 1.
I20260812 00:34:59.054252 24038 leader_election.cc:302] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506d7dfe91af4e75bdde8c8a73fedd11, 7e715bfb5c8848df92d3ce23d8c1d675; no voters:
I20260812 00:34:59.054577 24617 raft_consensus.cc:2781] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:34:59.054746 24617 raft_consensus.cc:686] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 LEADER]: Becoming Leader. State: Replica: 506d7dfe91af4e75bdde8c8a73fedd11, State: Running, Role: LEADER
I20260812 00:34:59.054740 24437 raft_consensus.cc:2454] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 506d7dfe91af4e75bdde8c8a73fedd11 in term 1.
I20260812 00:34:59.055076 24617 consensus_queue.cc:237] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } }
I20260812 00:34:59.058076 23546 catalog_manager.cc:5495] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66). New cstate: current_term: 1 leader_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "506d7dfe91af4e75bdde8c8a73fedd11" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 } health_report { overall_health: UNKNOWN } } }
I20260812 00:34:59.109216 24755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:59.102922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:39241 (local address 127.7.244.66:34303)
0812 00:34:59.103707 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:59.103727 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:59.103903 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:59.104231 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:59.104235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:59.104246 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:34:59.104560 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:59.104570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.105807 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.105812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:59.106961 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:59.106976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.107824 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.107831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:59.107983 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:59.108058 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:34:59.108531 (+ 473us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:34:59.108544 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 00:34:59.108617 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 00:34:59.108781 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":416,"threads_started":1}
I20260812 00:34:59.109367 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:59.102910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:34303 (local address 127.7.244.67:39241)
0812 00:34:59.103685 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:59.103691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:59.103811 (+ 120us) server_negotiation.cc:407] Connection header received
0812 00:34:59.104039 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:59.104047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:59.104116 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:59.104201 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:59.104739 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.105634 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.107125 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.107646 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.107768 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:59.108279 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:59.108418 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:59.108513 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:59.108681 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 00:34:59.108770 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":343,"threads_started":1}
I20260812 00:34:59.119916 23880 raft_consensus.cc:1261] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Refusing update from remote peer 506d7dfe91af4e75bdde8c8a73fedd11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:59.120666 24437 raft_consensus.cc:1261] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Refusing update from remote peer 506d7dfe91af4e75bdde8c8a73fedd11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:34:59.120684 24617 consensus_queue.cc:1035] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e715bfb5c8848df92d3ce23d8c1d675" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:59.121250 24729 consensus_queue.cc:1035] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a54d44392da41608e7fcb22095564fe" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 39241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:34:59.184269 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:59.166643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:51041 (local address 127.7.244.66:41907)
0812 00:34:59.166834 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:59.166841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:59.178453 (+ 11612us) server_negotiation.cc:407] Connection header received
0812 00:34:59.178988 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:59.178995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:59.179127 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:59.179185 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:34:59.179923 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.180737 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.182390 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.182831 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.182985 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:59.183511 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:34:59.183651 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:34:59.183743 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:59.183876 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 00:34:59.183963 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:34:59.184388 24777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:59.166550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:41907 (local address 127.7.244.67:51041)
0812 00:34:59.178305 (+ 11755us) negotiation.cc:107] Waiting for socket to connect
0812 00:34:59.178328 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:34:59.178778 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:34:59.179375 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:34:59.179378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:34:59.179388 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 00:34:59.179729 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:59.179738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.180884 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.180888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:59.182235 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:34:59.182250 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.183049 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.183056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:34:59.183254 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:59.183311 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 00:34:59.183756 (+ 445us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 00:34:59.183769 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 00:34:59.183839 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 00:34:59.184000 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11593,"thread_start_us":11931,"threads_started":1}
I20260812 00:34:59.225855 23555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:59.226097 23555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:34:59.229235 23555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 651b0b62d8d94975bfd9b25938dbba80: Sending DeleteTablet for 3 replicas of tablet 655140effa224573ae7fc7a5b0141bcd
I20260812 00:34:59.230159 23843 tablet_service.cc:1514] Processing DeleteTablet for tablet 655140effa224573ae7fc7a5b0141bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:59 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55430
I20260812 00:34:59.230244 24407 tablet_service.cc:1514] Processing DeleteTablet for tablet 655140effa224573ae7fc7a5b0141bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:59 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:43508
I20260812 00:34:59.230301 24128 tablet_service.cc:1514] Processing DeleteTablet for tablet 655140effa224573ae7fc7a5b0141bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:34:59 UTC) from {username='oryx', principal='oryx/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:52534
I20260812 00:34:59.230542 24698 tablet_replica.cc:330] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: stopping tablet replica
I20260812 00:34:59.230598 24702 tablet_replica.cc:330] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: stopping tablet replica
I20260812 00:34:59.230611 24700 tablet_replica.cc:330] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: stopping tablet replica
I20260812 00:34:59.230736 24698 raft_consensus.cc:2229] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:59.230772 24700 raft_consensus.cc:2229] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:34:59.230777 24702 raft_consensus.cc:2229] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:34:59.230959 24698 raft_consensus.cc:2258] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:59.230996 24702 raft_consensus.cc:2258] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:59.231125 24700 raft_consensus.cc:2258] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:34:59.232173 24700 ts_tablet_manager.cc:1905] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:59.232173 24702 ts_tablet_manager.cc:1905] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:59.232237 24698 ts_tablet_manager.cc:1905] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:34:59.237162 24698 ts_tablet_manager.cc:1918] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:59.237321 24698 log.cc:1196] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/655140effa224573ae7fc7a5b0141bcd
I20260812 00:34:59.237705 24698 ts_tablet_manager.cc:1939] T 655140effa224573ae7fc7a5b0141bcd P 7e715bfb5c8848df92d3ce23d8c1d675: Deleting consensus metadata
I20260812 00:34:59.238888 24700 ts_tablet_manager.cc:1918] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:59.238888 24702 ts_tablet_manager.cc:1918] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 00:34:59.238839 23523 catalog_manager.cc:4841] TS 7e715bfb5c8848df92d3ce23d8c1d675 (127.7.244.65:38439): tablet 655140effa224573ae7fc7a5b0141bcd (table test-table [id=6e7377ae8ebb46cdb206052f9642f807]) successfully deleted
I20260812 00:34:59.239033 24700 log.cc:1196] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/655140effa224573ae7fc7a5b0141bcd
I20260812 00:34:59.239046 24702 log.cc:1196] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/655140effa224573ae7fc7a5b0141bcd
I20260812 00:34:59.239390 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 23718
I20260812 00:34:59.239457 24700 ts_tablet_manager.cc:1939] T 655140effa224573ae7fc7a5b0141bcd P 506d7dfe91af4e75bdde8c8a73fedd11: Deleting consensus metadata
I20260812 00:34:59.239471 24702 ts_tablet_manager.cc:1939] T 655140effa224573ae7fc7a5b0141bcd P 6a54d44392da41608e7fcb22095564fe: Deleting consensus metadata
I20260812 00:34:59.240635 23523 catalog_manager.cc:4841] TS 6a54d44392da41608e7fcb22095564fe (127.7.244.67:39241): tablet 655140effa224573ae7fc7a5b0141bcd (table test-table [id=6e7377ae8ebb46cdb206052f9642f807]) successfully deleted
I20260812 00:34:59.241047 23522 catalog_manager.cc:4841] TS 506d7dfe91af4e75bdde8c8a73fedd11 (127.7.244.66:41907): tablet 655140effa224573ae7fc7a5b0141bcd (table test-table [id=6e7377ae8ebb46cdb206052f9642f807]) successfully deleted
Aug 12 00:34:59 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.65@KRBTEST.COM
I20260812 00:34:59.258759 24790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:59.240830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55442 (local address 127.7.244.65:38439)
0812 00:34:59.241466 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:59.241472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:59.241495 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:34:59.241630 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:59.241634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:59.241697 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:59.241783 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:59.242974 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.243760 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.244976 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.245439 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.245531 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:59.252887 (+ 7356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:59.252908 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:59.252928 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:59.252984 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:59.255859 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:59.256681 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:59.256690 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:59.256692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:59.256847 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:59.257151 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:59.257154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:59.257156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:59.257648 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:59.257879 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:59.258196 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 00:34:59.258406 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":303,"threads_started":1}
I20260812 00:34:59.259541 23966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:59.357982 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 23718
I20260812 00:34:59.389256 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 23994
Aug 12 00:34:59 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.66@KRBTEST.COM
I20260812 00:34:59.408386 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:59.390149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.7.244.66:41907)
0812 00:34:59.390311 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:59.390317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:59.390337 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:34:59.390517 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:59.390521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:59.390610 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:59.390691 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:59.392252 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.392983 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.395063 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.395531 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.395651 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:59.401095 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:59.401125 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:59.401149 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:59.401204 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:59.405252 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:59.406140 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:59.406151 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:59.406155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:59.406376 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:59.406738 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:59.406743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:59.406747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:59.407225 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:59.407526 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:59.407763 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 00:34:59.408009 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 00:34:59.409104 24260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:59.535179 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 23994
I20260812 00:34:59.569900 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 24277
Aug 12 00:34:59 dist-test-slave-gjw7 krb5kdc[23466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.244.67@KRBTEST.COM
I20260812 00:34:59.586334 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:34:59.570759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.7.244.67:39241)
0812 00:34:59.570952 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 00:34:59.570957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:34:59.570978 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:34:59.571043 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:34:59.571047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:34:59.571133 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:34:59.571233 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:34:59.572745 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.573627 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.575048 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:34:59.575460 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:34:59.575566 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:34:59.580321 (+ 4755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:34:59.580340 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:34:59.580360 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:34:59.580409 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:34:59.583410 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:59.584221 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:59.584228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:59.584230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:59.584416 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:34:59.584757 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:34:59.584761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:34:59.584763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:34:59.585163 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:34:59.585513 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0812 00:34:59.585844 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 00:34:59.586063 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:34:59.587009 24528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:59.733625 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 24277
I20260812 00:34:59.768296 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 23484
I20260812 00:34:59.769623 23645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:34:59.888782 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 23484
2026-08-12T00:34:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7856 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:35:00 dist-test-slave-gjw7 krb5kdc[24798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:35:00 dist-test-slave-gjw7 krb5kdc[24798](info): set up 2 sockets
Aug 12 00:35:00 dist-test-slave-gjw7 krb5kdc[24798](info): commencing operation
krb5kdc: starting...
W20260812 00:35:01.206890 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:35:01 dist-test-slave-gjw7 krb5kdc[24798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:35:01.493755 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:38301
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:41517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:38301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:01.862268 24843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:01.862731 24843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:01.862886 24843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:01.870635 24843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:35:01.870788 24843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:01.870872 24843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:01.870960 24843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:35:01.871030 24843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:35:01.884467 24843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:38301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:38301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:01.886255 24843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:01.887741 24843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:01.900961 24859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:01.901089 24856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:01.901116 24857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:01.902058 24843 server_base.cc:1034] running on GCE node
I20260812 00:35:01.902982 24843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:01.904636 24843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:01.905891 24843 hybrid_clock.cc:648] HybridClock initialized: now 1786494901905832 us; error 44 us; skew 500 ppm
I20260812 00:35:01.906284 24843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T00:35:02Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2086 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:35:02 dist-test-slave-gjw7 krb5kdc[24869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:35:02 dist-test-slave-gjw7 krb5kdc[24869](info): set up 2 sockets
Aug 12 00:35:02 dist-test-slave-gjw7 krb5kdc[24869](info): commencing operation
krb5kdc: starting...
W20260812 00:35:03.297959 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:35:03 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:35:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:35:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:35:03.577440 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:45309
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:45309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:03.942728 24893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:03.943092 24893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:03.943204 24893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:03.950766 24893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:35:03.950904 24893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:03.950963 24893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:03.951046 24893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:35:03.951103 24893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:35:03.961530 24893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:45309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:03.963018 24893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:03.964457 24893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:03.977015 24904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:03.977149 24906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:03.977599 24909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:03.978114 24893 server_base.cc:1034] running on GCE node
I20260812 00:35:03.978995 24893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:03.980659 24893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:03.981880 24893 hybrid_clock.cc:648] HybridClock initialized: now 1786494903981826 us; error 48 us; skew 500 ppm
I20260812 00:35:03.982257 24893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:03 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494903, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:03.985690 24893 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:35:03.987413 24893 webserver.cc:460] Webserver started at http://127.7.244.126:41081/ using document root <none> and password file <none>
I20260812 00:35:03.988233 24893 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:03.988359 24893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:03.988655 24893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:03.991997 24893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "af7bf3546c5e42e087c4b7da62cf5880"
format_stamp: "Formatted at 2026-08-12 00:35:03 on dist-test-slave-gjw7"
server_key: "7c3a5602d612c79a6eacac90c767487b"
server_key_iv: "8dafa97a6c5d372e4394c4921dec1844"
server_key_version: "encryptionkey@0"
I20260812 00:35:03.992877 24893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "af7bf3546c5e42e087c4b7da62cf5880"
format_stamp: "Formatted at 2026-08-12 00:35:03 on dist-test-slave-gjw7"
server_key: "7c3a5602d612c79a6eacac90c767487b"
server_key_iv: "8dafa97a6c5d372e4394c4921dec1844"
server_key_version: "encryptionkey@0"
I20260812 00:35:03.998960 24893 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 00:35:04.003269 24921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:04.004974 24893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 00:35:04.005191 24893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "af7bf3546c5e42e087c4b7da62cf5880"
format_stamp: "Formatted at 2026-08-12 00:35:03 on dist-test-slave-gjw7"
server_key: "7c3a5602d612c79a6eacac90c767487b"
server_key_iv: "8dafa97a6c5d372e4394c4921dec1844"
server_key_version: "encryptionkey@0"
I20260812 00:35:04.005491 24893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:04.040256 24893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:04.044637 24893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:04.044929 24893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:04.063843 24893 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:45309
I20260812 00:35:04.063874 25024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:45309 every 8 connection(s)
I20260812 00:35:04.065407 24893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:35:04.070187 25025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:35:04.070884 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 24893
I20260812 00:35:04.071125 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:35:04.071964 8145 external_mini_cluster.cc:1428] Setting key 56107c28fc38edb0448686baed4d6251
I20260812 00:35:04.086069 25025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Bootstrap starting.
Aug 12 00:35:04 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:35:04.090484 25025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Neither blocks nor log segments found. Creating new log.
I20260812 00:35:04.091740 25025 log.cc:824] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Log is configured to *not* fsync() on all Append() calls
I20260812 00:35:04.095533 25025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: No bootstrap required, opened a new log
I20260812 00:35:04.099438 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:04.074421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.7.244.126:45309)
0812 00:35:04.075380 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:04.075395 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:04.075443 (+ 48us) server_negotiation.cc:407] Connection header received
0812 00:35:04.077483 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:04.077518 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:04.078037 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:04.078451 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:04.080457 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:04.081959 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:04.083828 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:04.084633 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:04.084802 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:04.090062 (+ 5260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:04.090073 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:04.090093 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:04.090144 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:04.093908 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:04.094774 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:04.094784 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:04.094794 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:04.095047 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:04.095470 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:04.095474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:04.095477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:04.096346 (+ 869us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:04.096782 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:04.097245 (+ 463us) server_negotiation.cc:299] Negotiation successful
0812 00:35:04.097816 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":421,"threads_started":1}
I20260812 00:35:04.105967 25025 raft_consensus.cc:348] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } }
I20260812 00:35:04.106391 25025 raft_consensus.cc:374] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:35:04.106480 25025 raft_consensus.cc:729] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af7bf3546c5e42e087c4b7da62cf5880, State: Initialized, Role: FOLLOWER
I20260812 00:35:04.107045 25025 consensus_queue.cc:260] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } }
I20260812 00:35:04.107288 25025 raft_consensus.cc:388] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:35:04.107391 25025 raft_consensus.cc:482] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:35:04.107530 25025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:35:04.110294 25025 raft_consensus.cc:504] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } }
I20260812 00:35:04.110913 25025 leader_election.cc:302] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af7bf3546c5e42e087c4b7da62cf5880; no voters:
I20260812 00:35:04.111514 25025 leader_election.cc:288] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:35:04.111653 25033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:35:04.112439 25033 raft_consensus.cc:686] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 1 LEADER]: Becoming Leader. State: Replica: af7bf3546c5e42e087c4b7da62cf5880, State: Running, Role: LEADER
I20260812 00:35:04.113050 25033 consensus_queue.cc:237] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } }
I20260812 00:35:04.113919 25025 sys_catalog.cc:564] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:35:04.116254 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af7bf3546c5e42e087c4b7da62cf5880. Latest consensus state: current_term: 1 leader_uuid: "af7bf3546c5e42e087c4b7da62cf5880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } } }
I20260812 00:35:04.116364 25037 sys_catalog.cc:455] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af7bf3546c5e42e087c4b7da62cf5880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } } }
I20260812 00:35:04.116833 25037 sys_catalog.cc:458] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:04.116833 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:04.122689 25048 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:35:04.128512 25048 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:35:04.138823 25048 catalog_manager.cc:1347] Generated new cluster ID: b7932f3e3add4d778ce66e78a2218144
I20260812 00:35:04.138964 25048 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:35:04.163357 25048 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:35:04.165534 25048 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:35:04.177066 25048 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Generated new TSK 0
I20260812 00:35:04.178174 25048 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:35:04.249089 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:45309
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:04.607601 25088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:04.607924 25088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:04.608064 25088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:04.615782 25088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:04.615936 25088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:04.616091 25088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:35:04.626507 25088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:04.627928 25088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:04.629448 25088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:04.643833 25094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:04.643904 25096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:04.643869 25093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:04.644757 25088 server_base.cc:1034] running on GCE node
I20260812 00:35:04.645500 25088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:04.646529 25088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:04.647773 25088 hybrid_clock.cc:648] HybridClock initialized: now 1786494904647718 us; error 45 us; skew 500 ppm
I20260812 00:35:04.648128 25088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:04 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494904, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:04.655114 25088 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:35:04.656831 25088 webserver.cc:460] Webserver started at http://127.7.244.65:34013/ using document root <none> and password file <none>
I20260812 00:35:04.657697 25088 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:04.657826 25088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:04.658159 25088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:04.661391 25088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "d198dfed18a14d869078d60b5e7f2a2b"
format_stamp: "Formatted at 2026-08-12 00:35:04 on dist-test-slave-gjw7"
server_key: "b9f9a0c490f8fb928544d5a1e6d1dd91"
server_key_iv: "fbe18613701d629dcd1b2250fa1233df"
server_key_version: "encryptionkey@0"
I20260812 00:35:04.662256 25088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "d198dfed18a14d869078d60b5e7f2a2b"
format_stamp: "Formatted at 2026-08-12 00:35:04 on dist-test-slave-gjw7"
server_key: "b9f9a0c490f8fb928544d5a1e6d1dd91"
server_key_iv: "fbe18613701d629dcd1b2250fa1233df"
server_key_version: "encryptionkey@0"
I20260812 00:35:04.668094 25088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 00:35:04.672339 25112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:04.673945 25088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:35:04.674139 25088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "d198dfed18a14d869078d60b5e7f2a2b"
format_stamp: "Formatted at 2026-08-12 00:35:04 on dist-test-slave-gjw7"
server_key: "b9f9a0c490f8fb928544d5a1e6d1dd91"
server_key_iv: "fbe18613701d629dcd1b2250fa1233df"
server_key_version: "encryptionkey@0"
I20260812 00:35:04.674356 25088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:04.691969 25088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:04.696497 25088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:04.696815 25088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:04.697989 25088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:04.699548 25088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:04.699635 25088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:04.699770 25088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:04.699823 25088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:04.735812 25088 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:43017
I20260812 00:35:04.735862 25349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:43017 every 8 connection(s)
I20260812 00:35:04.737218 25088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:35:04.739092 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 25088
I20260812 00:35:04.739293 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:35:04.740126 8145 external_mini_cluster.cc:1428] Setting key 93d38aeebad2d1b8af6eff8bccfbf7bb
Aug 12 00:35:04 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494904, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:35:04.761583 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:04.740102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:33571 (local address 127.7.244.126:45309)
0812 00:35:04.740590 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:04.740595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:04.741861 (+ 1266us) server_negotiation.cc:407] Connection header received
0812 00:35:04.744385 (+ 2524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:04.744390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:04.744466 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:04.744560 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:04.746788 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:04.747564 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:04.749821 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:04.750263 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:04.750382 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:04.754848 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:04.754857 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:04.754863 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:04.754910 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:04.758096 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:04.759019 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:04.759026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:04.759029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:04.759187 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:04.759634 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:04.759640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:04.759643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:04.760029 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:04.760174 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:04.760957 (+ 783us) server_negotiation.cc:299] Negotiation successful
0812 00:35:04.761211 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":260,"threads_started":1}
I20260812 00:35:04.762485 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:04.740778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:33571)
0812 00:35:04.741683 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:04.741733 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:04.744031 (+ 2298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:04.744877 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:04.744890 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:04.745710 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:04.746553 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:04.746583 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:04.747747 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:04.747752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:04.749650 (+ 1898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:04.749669 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:04.750516 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:04.750523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:04.750829 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:04.751613 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:04.751643 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:04.754531 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:04.758283 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:04.758293 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:04.758311 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:04.758848 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:04.759284 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:04.759287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:04.759289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:04.759500 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:04.760142 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:04.760149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:04.760518 (+ 369us) client_negotiation.cc:769] Sending connection context
0812 00:35:04.760764 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 00:35:04.761213 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":339,"threads_started":1}
I20260812 00:35:04.764217 25351 heartbeater.cc:347] Connected to a master server at 127.7.244.126:45309
I20260812 00:35:04.764602 25351 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:04.765688 25351 heartbeater.cc:511] Master 127.7.244.126:45309 requested a full tablet report, sending...
I20260812 00:35:04.768258 24954 ts_manager.cc:194] Registered new tserver with Master: d198dfed18a14d869078d60b5e7f2a2b (127.7.244.65:43017)
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:35:04.769964 24954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:33571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:35:04.804054 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:45309
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:05.162464 25362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:05.162796 25362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:05.162899 25362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:05.170871 25362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:05.171015 25362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:05.171137 25362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:35:05.181874 25362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:05.183225 25362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:05.184630 25362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:05.198800 25379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:05.198907 25373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:05.198830 25375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:05.199783 25362 server_base.cc:1034] running on GCE node
I20260812 00:35:05.200462 25362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:05.201607 25362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:05.202883 25362 hybrid_clock.cc:648] HybridClock initialized: now 1786494905202797 us; error 45 us; skew 500 ppm
I20260812 00:35:05.203246 25362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:05 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494905, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:05.206775 25362 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:35:05.208534 25362 webserver.cc:460] Webserver started at http://127.7.244.66:43557/ using document root <none> and password file <none>
I20260812 00:35:05.209427 25362 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:05.209559 25362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:05.209908 25362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:05.213276 25362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "13e09ee2d96549cbbc51ef675cce21c4"
format_stamp: "Formatted at 2026-08-12 00:35:05 on dist-test-slave-gjw7"
server_key: "2766000e45424d3d585aaaa34c5bc21e"
server_key_iv: "90fba590634ba645f17d36c1ae67a4dc"
server_key_version: "encryptionkey@0"
I20260812 00:35:05.214354 25362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "13e09ee2d96549cbbc51ef675cce21c4"
format_stamp: "Formatted at 2026-08-12 00:35:05 on dist-test-slave-gjw7"
server_key: "2766000e45424d3d585aaaa34c5bc21e"
server_key_iv: "90fba590634ba645f17d36c1ae67a4dc"
server_key_version: "encryptionkey@0"
I20260812 00:35:05.221014 25362 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:35:05.225529 25398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:05.227288 25362 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:35:05.227509 25362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "13e09ee2d96549cbbc51ef675cce21c4"
format_stamp: "Formatted at 2026-08-12 00:35:05 on dist-test-slave-gjw7"
server_key: "2766000e45424d3d585aaaa34c5bc21e"
server_key_iv: "90fba590634ba645f17d36c1ae67a4dc"
server_key_version: "encryptionkey@0"
I20260812 00:35:05.227751 25362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:05.264631 25362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:05.269430 25362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:05.269748 25362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:05.270905 25362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:05.272440 25362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:05.272529 25362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:05.272672 25362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:05.272728 25362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:05.305328 25362 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:34823
I20260812 00:35:05.305415 25650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:34823 every 8 connection(s)
I20260812 00:35:05.306762 25362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:35:05.312186 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 25362
I20260812 00:35:05.312392 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:35:05.312997 8145 external_mini_cluster.cc:1428] Setting key 0d4c2a246f686717727080896671e834
Aug 12 00:35:05 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494905, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:35:05.331658 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:05.309582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:54863 (local address 127.7.244.126:45309)
0812 00:35:05.310069 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:05.310074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:05.311124 (+ 1050us) server_negotiation.cc:407] Connection header received
0812 00:35:05.313797 (+ 2673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:05.313805 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:05.313916 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:05.314063 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:05.316149 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:05.316933 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:05.319364 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:05.319819 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:05.319942 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:05.324722 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:05.324730 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:05.324735 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:05.324782 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:05.327797 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:05.328904 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:05.328912 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:05.328916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:05.329091 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:05.329670 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:05.329675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:05.329678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:05.330069 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:05.330218 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:05.330985 (+ 767us) server_negotiation.cc:299] Negotiation successful
0812 00:35:05.331274 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":229,"threads_started":1}
I20260812 00:35:05.332929 25660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:05.310172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:54863)
0812 00:35:05.310951 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:05.310994 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:05.313337 (+ 2343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:05.314388 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:05.314398 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:05.315031 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:05.315894 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:05.315963 (+ 69us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:05.317129 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:05.317135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:05.319120 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:05.319140 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:05.320106 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:05.320113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:05.320421 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:05.321256 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:05.321329 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:05.324360 (+ 3031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:05.328002 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:05.328011 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:05.328027 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:05.328715 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:05.329209 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:05.329213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:05.329215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:05.329516 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:05.330186 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:05.330193 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:05.330592 (+ 399us) client_negotiation.cc:769] Sending connection context
0812 00:35:05.330834 (+ 242us) client_negotiation.cc:240] Negotiation successful
0812 00:35:05.331296 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":298,"threads_started":1}
I20260812 00:35:05.334784 25652 heartbeater.cc:347] Connected to a master server at 127.7.244.126:45309
I20260812 00:35:05.335130 25652 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:05.335949 25652 heartbeater.cc:511] Master 127.7.244.126:45309 requested a full tablet report, sending...
I20260812 00:35:05.337687 24954 ts_manager.cc:194] Registered new tserver with Master: 13e09ee2d96549cbbc51ef675cce21c4 (127.7.244.66:34823)
I20260812 00:35:05.338598 24954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:54863
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:35:05.378177 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:45309
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:05.735567 25664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:05.735879 25664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:05.736027 25664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:05.743726 25664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:05.743865 25664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:05.744022 25664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:35:05.754457 25664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:05.756258 25664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:05.758214 25664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:35:05.773811 25351 heartbeater.cc:503] Master 127.7.244.126:45309 was elected leader, sending a full tablet report...
W20260812 00:35:05.775995 25679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:05.778488 25682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:05.778787 25677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:05.785458 25664 server_base.cc:1034] running on GCE node
I20260812 00:35:05.786717 25664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:05.788246 25664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:05.789557 25664 hybrid_clock.cc:648] HybridClock initialized: now 1786494905789475 us; error 68 us; skew 500 ppm
I20260812 00:35:05.790096 25664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:05 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494905, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:05.794884 25664 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:35:05.797152 25664 webserver.cc:460] Webserver started at http://127.7.244.67:40491/ using document root <none> and password file <none>
I20260812 00:35:05.798218 25664 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:05.798375 25664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:05.798715 25664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:05.803762 25664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "18ef124e6cea42189a14b115d34d9c34"
format_stamp: "Formatted at 2026-08-12 00:35:05 on dist-test-slave-gjw7"
server_key: "611f8ce9ebd5895d40387b015db31cfc"
server_key_iv: "41563e876ff7d9904d6288bf1d05eea9"
server_key_version: "encryptionkey@0"
I20260812 00:35:05.804978 25664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "18ef124e6cea42189a14b115d34d9c34"
format_stamp: "Formatted at 2026-08-12 00:35:05 on dist-test-slave-gjw7"
server_key: "611f8ce9ebd5895d40387b015db31cfc"
server_key_iv: "41563e876ff7d9904d6288bf1d05eea9"
server_key_version: "encryptionkey@0"
I20260812 00:35:05.812726 25664 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260812 00:35:05.817598 25700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:05.819319 25664 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 00:35:05.819504 25664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "18ef124e6cea42189a14b115d34d9c34"
format_stamp: "Formatted at 2026-08-12 00:35:05 on dist-test-slave-gjw7"
server_key: "611f8ce9ebd5895d40387b015db31cfc"
server_key_iv: "41563e876ff7d9904d6288bf1d05eea9"
server_key_version: "encryptionkey@0"
I20260812 00:35:05.819732 25664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:05.835853 25664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:05.840939 25664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:05.841236 25664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:05.842427 25664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:05.843897 25664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:05.843986 25664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:05.844115 25664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:05.844173 25664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:05.875201 25664 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:41309
I20260812 00:35:05.875250 25928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:41309 every 8 connection(s)
I20260812 00:35:05.876667 25664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:35:05.879294 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 25664
I20260812 00:35:05.879527 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:35:05.880187 8145 external_mini_cluster.cc:1428] Setting key 4b35a6c3c1ffa3776a12512b779936d6
Aug 12 00:35:05 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494905, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:35:05.900856 25940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:05.881558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:37499 (local address 127.7.244.126:45309)
0812 00:35:05.882287 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:05.882292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:05.882313 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:35:05.884813 (+ 2500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:05.884819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:05.884925 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:05.885026 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:05.887088 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:05.887912 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:05.889979 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:05.890457 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:05.890582 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:05.894505 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:05.894512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:05.894515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:05.894554 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:05.897449 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:05.898313 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:05.898324 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:05.898326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:05.898473 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:05.898954 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:05.898958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:05.898960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:05.899252 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:05.899407 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:05.900217 (+ 810us) server_negotiation.cc:299] Negotiation successful
0812 00:35:05.900475 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":468,"threads_started":1}
I20260812 00:35:05.901906 25938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:05.880698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:37499)
0812 00:35:05.881992 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:05.882064 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:05.884436 (+ 2372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:05.885395 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:05.885418 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:05.886165 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:05.886869 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:05.886890 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:05.888098 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:05.888104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:05.889709 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:05.889728 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:05.890727 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:05.890736 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:05.891000 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:05.891738 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:05.891763 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:05.894206 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:05.897625 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:05.897635 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:05.897652 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:05.898169 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:05.898591 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:05.898597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:05.898601 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:05.898831 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:05.899389 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:05.899398 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:05.899792 (+ 394us) client_negotiation.cc:769] Sending connection context
0812 00:35:05.900072 (+ 280us) client_negotiation.cc:240] Negotiation successful
0812 00:35:05.900696 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"thread_start_us":669,"threads_started":1}
I20260812 00:35:05.903409 25929 heartbeater.cc:347] Connected to a master server at 127.7.244.126:45309
I20260812 00:35:05.903746 25929 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:05.904547 25929 heartbeater.cc:511] Master 127.7.244.126:45309 requested a full tablet report, sending...
I20260812 00:35:05.906152 24954 ts_manager.cc:194] Registered new tserver with Master: 18ef124e6cea42189a14b115d34d9c34 (127.7.244.67:41309)
I20260812 00:35:05.906996 24954 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:37499
I20260812 00:35:05.909883 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab.
Entry for principal oryx/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab.
WARNING: no policy specified for oryx/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab.
Entry for principal oryx/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab.
WARNING: no policy specified for oryx/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab.
Entry for principal oryx/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab.
I20260812 00:35:06.084527 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 25088
Aug 12 00:35:06 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:35:06.103044 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:06.085876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39536 (local address 127.7.244.65:43017)
0812 00:35:06.086834 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:06.086845 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:06.086883 (+ 38us) server_negotiation.cc:407] Connection header received
0812 00:35:06.086999 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:06.087010 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:06.087304 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:06.087453 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:06.088945 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:06.090144 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:06.091373 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:06.092001 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:06.092109 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:06.096512 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:06.096531 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:06.096549 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:06.096610 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:06.099819 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:06.100670 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:06.100679 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:06.100681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:06.100836 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:06.101244 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:06.101251 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:06.101255 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:06.101727 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:06.102014 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:06.102387 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 00:35:06.102641 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":446,"threads_started":1}
I20260812 00:35:06.104467 25334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:06.188571 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 25088
I20260812 00:35:06.214268 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 25362
Aug 12 00:35:06 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:35:06.232786 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:06.215281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.7.244.66:34823)
0812 00:35:06.215868 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:06.215875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:06.215905 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:35:06.215991 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:06.215998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:06.216243 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:06.216389 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:06.217724 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:06.218794 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:06.220190 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:06.220845 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:06.220965 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:06.225976 (+ 5011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:06.225999 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:06.226018 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:06.226075 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:06.229581 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:06.230480 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:06.230490 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:06.230492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:06.230672 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:06.231070 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:06.231074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:06.231076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:06.231540 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:06.231848 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:06.232096 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 00:35:06.232366 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":317,"threads_started":1}
I20260812 00:35:06.234225 25644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:06.318444 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 25362
I20260812 00:35:06.349366 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 25664
Aug 12 00:35:06 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:35:06.367383 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:06.350387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56912 (local address 127.7.244.67:41309)
0812 00:35:06.350952 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:06.350960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:06.350994 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:35:06.351060 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:06.351066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:06.351291 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:06.351435 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:06.352929 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:06.354140 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:06.355488 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:06.356094 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:06.356203 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:06.361121 (+ 4918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:06.361140 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:06.361159 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:06.361205 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:06.364434 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:06.365318 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:06.365330 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:06.365333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:06.365509 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:06.365858 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:06.365861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:06.365863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:06.366252 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:06.366499 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:06.366805 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 00:35:06.367070 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":236,"threads_started":1}
I20260812 00:35:06.368762 25916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:06.452696 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 25664
I20260812 00:35:06.475203 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 24893
I20260812 00:35:06.476624 25017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:06.582032 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 24893
I20260812 00:35:06.612159 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:45309
--webserver_interface=127.7.244.126
--webserver_port=41081
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:45309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:06.971598 25974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:06.971988 25974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:06.972126 25974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:06.980115 25974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:35:06.980280 25974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:06.980367 25974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:06.980445 25974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:35:06.980509 25974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:35:06.991137 25974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:45309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=41081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:06.992635 25974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:06.994052 25974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:07.006897 25989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:07.006832 25991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:07.006870 25993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:07.008347 25974 server_base.cc:1034] running on GCE node
I20260812 00:35:07.009379 25974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:07.011059 25974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:07.012329 25974 hybrid_clock.cc:648] HybridClock initialized: now 1786494907012272 us; error 54 us; skew 500 ppm
I20260812 00:35:07.012737 25974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:07 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494907, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:07.016283 25974 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:35:07.018110 25974 webserver.cc:460] Webserver started at http://127.7.244.126:41081/ using document root <none> and password file <none>
I20260812 00:35:07.018904 25974 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:07.019048 25974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:07.025698 25974 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:35:07.029171 26004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:07.030916 25974 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:35:07.031139 25974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "af7bf3546c5e42e087c4b7da62cf5880"
format_stamp: "Formatted at 2026-08-12 00:35:03 on dist-test-slave-gjw7"
server_key: "7c3a5602d612c79a6eacac90c767487b"
server_key_iv: "8dafa97a6c5d372e4394c4921dec1844"
server_key_version: "encryptionkey@0"
I20260812 00:35:07.032249 25974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:07.071856 25974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:07.076817 25974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:07.077108 25974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:07.096571 25974 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:45309
I20260812 00:35:07.096591 26109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:45309 every 8 connection(s)
I20260812 00:35:07.098270 25974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:35:07.101820 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 25974
I20260812 00:35:07.102636 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:43017
--local_ip_for_outbound_sockets=127.7.244.65
--tserver_master_addrs=127.7.244.126:45309
--webserver_port=34013
--webserver_interface=127.7.244.65
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.244.65
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:35:07.105819 26110 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:35:07.109535 26110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Bootstrap starting.
I20260812 00:35:07.130134 26110 log.cc:824] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Log is configured to *not* fsync() on all Append() calls
I20260812 00:35:07.141079 26110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:35:07.141772 26110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Bootstrap complete.
I20260812 00:35:07.151402 26110 raft_consensus.cc:348] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } }
I20260812 00:35:07.152429 26110 raft_consensus.cc:729] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: af7bf3546c5e42e087c4b7da62cf5880, State: Initialized, Role: FOLLOWER
I20260812 00:35:07.153178 26110 consensus_queue.cc:260] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } }
I20260812 00:35:07.153481 26110 raft_consensus.cc:388] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:35:07.153605 26110 raft_consensus.cc:482] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:35:07.153781 26110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:35:07.158092 26110 raft_consensus.cc:504] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } }
I20260812 00:35:07.158812 26110 leader_election.cc:302] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af7bf3546c5e42e087c4b7da62cf5880; no voters:
I20260812 00:35:07.159916 26110 leader_election.cc:288] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 00:35:07.160089 26116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 00:35:07.161586 26116 raft_consensus.cc:686] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [term 2 LEADER]: Becoming Leader. State: Replica: af7bf3546c5e42e087c4b7da62cf5880, State: Running, Role: LEADER
I20260812 00:35:07.162410 26116 consensus_queue.cc:237] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } }
I20260812 00:35:07.162585 26110 sys_catalog.cc:564] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:35:07.165123 26121 sys_catalog.cc:455] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "af7bf3546c5e42e087c4b7da62cf5880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } } }
I20260812 00:35:07.165627 26121 sys_catalog.cc:458] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:07.165549 26124 sys_catalog.cc:455] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af7bf3546c5e42e087c4b7da62cf5880. Latest consensus state: current_term: 2 leader_uuid: "af7bf3546c5e42e087c4b7da62cf5880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af7bf3546c5e42e087c4b7da62cf5880" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45309 } } }
I20260812 00:35:07.165845 26124 sys_catalog.cc:458] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:07.173588 26135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:35:07.180681 26135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:35:07.182353 26135 catalog_manager.cc:1259] Loaded cluster ID: b7932f3e3add4d778ce66e78a2218144
I20260812 00:35:07.182551 26135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:35:07.186753 26135 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:35:07.188882 26135 catalog_manager.cc:5879] T 00000000000000000000000000000000 P af7bf3546c5e42e087c4b7da62cf5880: Loaded TSK: 0
I20260812 00:35:07.190610 26135 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:35:07.479135 26112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:07.479492 26112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:07.479635 26112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:07.487342 26112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:07.487497 26112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:07.487658 26112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:35:07.498507 26112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.65.keytab
--principal=oryx/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:43017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=34013
--tserver_master_addrs=127.7.244.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:07.500020 26112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:07.501513 26112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:07.517591 26175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:07.517591 26177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:07.521651 26183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:07.523721 26112 server_base.cc:1034] running on GCE node
I20260812 00:35:07.524488 26112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:07.525668 26112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:07.526921 26112 hybrid_clock.cc:648] HybridClock initialized: now 1786494907526850 us; error 65 us; skew 500 ppm
I20260812 00:35:07.527313 26112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:07 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494907, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:07.530737 26112 init.cc:377] Logged in from keytab as oryx/127.7.244.65@KRBTEST.COM (short username oryx)
I20260812 00:35:07.532421 26112 webserver.cc:460] Webserver started at http://127.7.244.65:34013/ using document root <none> and password file <none>
I20260812 00:35:07.533270 26112 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:07.533456 26112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:07.540159 26112 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 00:35:07.543607 26194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:07.545148 26112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:35:07.545380 26112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "d198dfed18a14d869078d60b5e7f2a2b"
format_stamp: "Formatted at 2026-08-12 00:35:04 on dist-test-slave-gjw7"
server_key: "b9f9a0c490f8fb928544d5a1e6d1dd91"
server_key_iv: "fbe18613701d629dcd1b2250fa1233df"
server_key_version: "encryptionkey@0"
I20260812 00:35:07.546586 26112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:07.565593 26112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:07.570421 26112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:07.570699 26112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:07.571823 26112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:07.573338 26112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:07.573446 26112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:07.573559 26112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:07.573637 26112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:07.617720 26112 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:43017
I20260812 00:35:07.617805 26434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:43017 every 8 connection(s)
I20260812 00:35:07.619345 26112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:35:07.623980 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 26112
I20260812 00:35:07.624603 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:34823
--local_ip_for_outbound_sockets=127.7.244.66
--tserver_master_addrs=127.7.244.126:45309
--webserver_port=43557
--webserver_interface=127.7.244.66
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.244.66
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:35:07 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:07 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 00:35:07.998505 26443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:07.998853 26443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:07.999002 26443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:08.007038 26443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:08.007195 26443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:08.007352 26443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:35:08.017765 26443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.66.keytab
--principal=oryx/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:34823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=43557
--tserver_master_addrs=127.7.244.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:08.019243 26443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:08.020690 26443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:08.038378 26453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:08.038462 26455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:08.039053 26462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:08.039747 26443 server_base.cc:1034] running on GCE node
I20260812 00:35:08.040500 26443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:08.041699 26443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:08.042992 26443 hybrid_clock.cc:648] HybridClock initialized: now 1786494908042925 us; error 42 us; skew 500 ppm
I20260812 00:35:08.043416 26443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494908, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:08.047389 26443 init.cc:377] Logged in from keytab as oryx/127.7.244.66@KRBTEST.COM (short username oryx)
I20260812 00:35:08.049489 26443 webserver.cc:460] Webserver started at http://127.7.244.66:43557/ using document root <none> and password file <none>
I20260812 00:35:08.050369 26443 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:08.050534 26443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:08.058125 26443 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 00:35:08.062032 26473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:08.064177 26443 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:35:08.064394 26443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "13e09ee2d96549cbbc51ef675cce21c4"
format_stamp: "Formatted at 2026-08-12 00:35:05 on dist-test-slave-gjw7"
server_key: "2766000e45424d3d585aaaa34c5bc21e"
server_key_iv: "90fba590634ba645f17d36c1ae67a4dc"
server_key_version: "encryptionkey@0"
I20260812 00:35:08.065582 26443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:08.105167 26443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:08.111306 26443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:08.111603 26443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:08.113111 26443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:08.114823 26443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:08.114946 26443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:08.115067 26443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:08.115139 26443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:08.152546 26443 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:34823
I20260812 00:35:08.152590 26721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:34823 every 8 connection(s)
I20260812 00:35:08.154300 26443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:35:08.155967 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 26443
I20260812 00:35:08.156569 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:41309
--local_ip_for_outbound_sockets=127.7.244.67
--tserver_master_addrs=127.7.244.126:45309
--webserver_port=40491
--webserver_interface=127.7.244.67
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.244.67
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 00:35:08.527541 26731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:08.527863 26731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:08.528012 26731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:08.535635 26731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:08.535784 26731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:08.535962 26731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:35:08.547120 26731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/krb5kdc/oryx_127.7.244.67.keytab
--principal=oryx/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:41309
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=40491
--tserver_master_addrs=127.7.244.126:45309
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:08.548727 26731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:08.550271 26731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:08.567782 26744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:08.572314 26742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:08.573117 26749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:08.573812 26731 server_base.cc:1034] running on GCE node
I20260812 00:35:08.574592 26731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:08.575764 26731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:08.577028 26731 hybrid_clock.cc:648] HybridClock initialized: now 1786494908576959 us; error 61 us; skew 500 ppm
I20260812 00:35:08.577426 26731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494908, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:08.580896 26731 init.cc:377] Logged in from keytab as oryx/127.7.244.67@KRBTEST.COM (short username oryx)
I20260812 00:35:08.587112 26731 webserver.cc:460] Webserver started at http://127.7.244.67:40491/ using document root <none> and password file <none>
I20260812 00:35:08.587922 26731 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:08.588081 26731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:08.594141 26731 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 00:35:08.597050 26760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:08.598611 26731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 00:35:08.598875 26731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "18ef124e6cea42189a14b115d34d9c34"
format_stamp: "Formatted at 2026-08-12 00:35:05 on dist-test-slave-gjw7"
server_key: "611f8ce9ebd5895d40387b015db31cfc"
server_key_iv: "41563e876ff7d9904d6288bf1d05eea9"
server_key_version: "encryptionkey@0"
I20260812 00:35:08.600184 26731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:08.615265 26731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:08.620965 26731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:08.621440 26731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:08.629987 26731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:08.631835 26731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:08.631953 26731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:08.632117 26731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:08.632192 26731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:08.677946 26731 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:41309
I20260812 00:35:08.677997 26991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:41309 every 8 connection(s)
I20260812 00:35:08.679715 26731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:35:08.685439 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 26731
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:08.726022 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:08.687206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.7.244.126:45309)
0812 00:35:08.699091 (+ 11885us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:08.699099 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:08.699123 (+ 24us) server_negotiation.cc:407] Connection header received
0812 00:35:08.699202 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:08.699208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:08.699296 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:08.699437 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:08.703550 (+ 4113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.705136 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.707180 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.708862 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.709016 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:08.715993 (+ 6977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:08.716009 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:08.716037 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:08.716092 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:08.720409 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:08.721740 (+ 1331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:08.721751 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:08.721762 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:08.722057 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:08.722545 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:08.722551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:08.722554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:08.723289 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:08.723696 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:08.724208 (+ 512us) server_negotiation.cc:299] Negotiation successful
0812 00:35:08.724786 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11732,"thread_start_us":10403,"threads_started":1}
W20260812 00:35:08.957660 26441 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.322s user 0.004s sys 0.011s
W20260812 00:35:08.959787 26439 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:07.622553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:44759 (local address 127.7.244.126:45309)
0812 00:35:07.623276 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:07.623289 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:07.623632 (+ 343us) server_negotiation.cc:407] Connection header received
0812 00:35:07.626248 (+ 2616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:07.626272 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:07.626848 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:07.627317 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:07.628998 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:07.630914 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:07.632799 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:07.633833 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:07.634043 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:08.959381 (+1325338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:44759: BlockingRecv error: recv got EOF from 127.7.244.65:44759 (error 108)
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":283,"threads_started":1}
I20260812 00:35:08.961218 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:07.622540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:44759)
0812 00:35:07.623403 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:07.623451 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:07.625466 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:07.627469 (+ 2003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:07.627478 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:07.628072 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:07.628733 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:07.628756 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:07.631135 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:07.631139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:07.632611 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:07.632629 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:07.634097 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:07.634103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:07.634367 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:07.635192 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:07.635225 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:08.958932 (+1323707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":350,"threads_started":1}
W20260812 00:35:08.963125 26435 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:45309 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.244.126:45309: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:08.978287 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:08.964073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:58197)
0812 00:35:08.964393 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:08.964414 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:08.964571 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:08.964989 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:08.964995 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:08.965612 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:08.965891 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:08.965920 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.967641 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.967648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:08.969609 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:08.969637 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.971052 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.971061 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:08.971279 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:08.972206 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:08.972234 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:08.977884 (+ 5650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173}
W20260812 00:35:08.978533 26439 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:08.964165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:58197 (local address 127.7.244.126:45309)
0812 00:35:08.964376 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:08.964383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:08.964504 (+ 121us) server_negotiation.cc:407] Connection header received
0812 00:35:08.964702 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:08.964708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:08.964802 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:08.964949 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:08.966082 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.967399 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.969842 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.970789 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.970965 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:08.978219 (+ 7254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:58197: BlockingRecv error: recv got EOF from 127.7.244.65:58197 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:08 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:08.994297 26441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:08.980555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:59535)
0812 00:35:08.980784 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:08.980811 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:08.980977 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:08.981456 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:08.981461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:08.982006 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:08.982278 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:08.982292 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.983968 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.983974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:08.985790 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:08.985811 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.986978 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.986984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:08.987203 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:08.988055 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:08.988083 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:08.993923 (+ 5840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 00:35:08.994591 26439 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:08.980323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:59535 (local address 127.7.244.126:45309)
0812 00:35:08.980919 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:08.980927 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:08.980954 (+ 27us) server_negotiation.cc:407] Connection header received
0812 00:35:08.981129 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:08.981135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:08.981252 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:08.981408 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:08.982450 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.983770 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.985975 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.986781 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.986940 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:08.994250 (+ 7310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:59535: BlockingRecv error: recv got EOF from 127.7.244.65:59535 (error 108)
Metrics: {"server-negotiator.queue_time_us":434}
W20260812 00:35:08.994969 26435 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:35:09.339956 26729 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.168s user 0.009s sys 0.008s
W20260812 00:35:09.341619 26727 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:08.159585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:45225 (local address 127.7.244.126:45309)
0812 00:35:08.160119 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:08.160124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:08.161859 (+ 1735us) server_negotiation.cc:407] Connection header received
0812 00:35:08.164541 (+ 2682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:08.164548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:08.164646 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:08.164729 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:08.167384 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.168234 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.170148 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.170686 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.170832 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.341138 (+1170306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:45225: BlockingRecv error: recv got EOF from 127.7.244.66:45225 (error 108)
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":268,"threads_started":1}
I20260812 00:35:09.341722 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:08.160561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:45225)
0812 00:35:08.161657 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:08.161717 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:08.164121 (+ 2404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:08.165426 (+ 1305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:08.165439 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:08.166229 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:08.167088 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:08.167119 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.168459 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.168463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:08.169930 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:08.169949 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.170949 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.170958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:08.171208 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:08.172033 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:08.172065 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:09.340744 (+1168679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":398,"threads_started":1}
W20260812 00:35:09.342911 26722 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:45309 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.244.126:45309: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:09.355747 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:09.343779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:60617)
0812 00:35:09.344026 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:09.344047 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:09.344163 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:09.344602 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:09.344605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:09.344981 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:09.345161 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.345171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.347048 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.347056 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.348569 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.348594 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.349995 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.350002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.350140 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.350939 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:09.350959 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:09.355360 (+ 4401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 00:35:09.355777 26727 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:09.343932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:60617 (local address 127.7.244.126:45309)
0812 00:35:09.344179 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:09.344187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:09.344239 (+ 52us) server_negotiation.cc:407] Connection header received
0812 00:35:09.344342 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:09.344349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:09.344444 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:09.344559 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:09.345378 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.346772 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.348855 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.349764 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.349936 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.355583 (+ 5647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:60617: BlockingRecv error: recv got EOF from 127.7.244.66:60617 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:09.368124 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:09.356940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:33113)
0812 00:35:09.357139 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:09.357163 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:09.357344 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:09.357692 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:09.357698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:09.358231 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:09.358541 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.358559 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.359754 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.359760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.361636 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.361660 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.362719 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.362724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.362896 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.363626 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:09.363645 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:09.367741 (+ 4096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 00:35:09.368242 26727 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:09.357165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:33113 (local address 127.7.244.126:45309)
0812 00:35:09.357366 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:09.357370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:09.357389 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:35:09.357470 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:09.357474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:09.357539 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:09.357620 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:09.358768 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.359566 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.361809 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.362369 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.362497 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.368005 (+ 5508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:33113: BlockingRecv error: recv got EOF from 127.7.244.66:33113 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 00:35:09.368836 26722 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:35:09.744143 27000 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.029s user 0.003s sys 0.012s
W20260812 00:35:09.745494 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:08.684446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:38137 (local address 127.7.244.126:45309)
0812 00:35:08.697881 (+ 13435us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:08.697888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:08.703438 (+ 5550us) server_negotiation.cc:407] Connection header received
0812 00:35:08.705889 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:08.705897 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:08.706018 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:08.706173 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:08.708693 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.709765 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.712731 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.713173 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.713735 (+ 562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.745217 (+1031482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:38137: BlockingRecv error: recv got EOF from 127.7.244.67:38137 (error 108)
Metrics: {"server-negotiator.queue_time_us":13287,"thread_start_us":14359,"threads_started":1}
I20260812 00:35:09.745990 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:08.683694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:38137)
0812 00:35:08.702804 (+ 19110us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:08.702875 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:08.705489 (+ 2614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:08.706541 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:08.706554 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:08.707426 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:08.708450 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:08.708484 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.709989 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.709997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:08.712486 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:08.712519 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:08.713502 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:08.713512 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:08.713845 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:08.714577 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:08.714601 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:09.744959 (+1030358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18754,"thread_start_us":18476,"threads_started":1}
W20260812 00:35:09.746999 26994 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:45309 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.244.126:45309: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:09.762251 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:09.747693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:35871)
0812 00:35:09.747893 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:09.747910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:09.748013 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:09.748326 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:09.748329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:09.748681 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:09.748861 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.748871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.750205 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.750211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.751596 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.751615 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.752380 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.752385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.752505 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.758269 (+ 5764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:09.758295 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:09.761935 (+ 3640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260812 00:35:09.762514 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:09.747850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:35871 (local address 127.7.244.126:45309)
0812 00:35:09.748009 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:09.748013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:09.748032 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:35:09.748120 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:09.748124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:09.748186 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:09.748271 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:09.749049 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.750039 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.751739 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.752223 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.752324 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.762197 (+ 9873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:35871: BlockingRecv error: recv got EOF from 127.7.244.67:35871 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:09.771556 27000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:09.763168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:53783)
0812 00:35:09.763359 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:09.763380 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:09.763501 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:09.763878 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:09.763882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:09.764330 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:09.764509 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.764519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.765576 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.765581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.766551 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.766564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.767221 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.767224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.767370 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.767959 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:09.767977 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:09.771332 (+ 3355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 00:35:09.771868 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:09.763318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:53783 (local address 127.7.244.126:45309)
0812 00:35:09.763597 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:09.763603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:09.763622 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:35:09.763668 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:09.763672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:09.763729 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:09.763826 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:09.764641 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.765456 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.766665 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.767111 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.767200 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:09.771594 (+ 4394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:53783: BlockingRecv error: recv got EOF from 127.7.244.67:53783 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
W20260812 00:35:09.771907 26994 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:09 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:10.006677 27009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:09.995729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:56171)
0812 00:35:09.996382 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:09.996403 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:09.996556 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:09.996929 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:09.996934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:09.997433 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:09.997589 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.997598 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.998604 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.998609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:09.999579 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:09.999597 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.000351 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.000358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:10.000522 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:10.001285 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:10.001356 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:10.006392 (+ 5036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":369,"threads_started":1}
W20260812 00:35:10.006925 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:09.995908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:56171 (local address 127.7.244.126:45309)
0812 00:35:09.996168 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:09.996174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:09.996505 (+ 331us) server_negotiation.cc:407] Connection header received
0812 00:35:09.996679 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:09.996685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:09.996780 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:09.996865 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:09.997747 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:09.998445 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:09.999737 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.000158 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.000275 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:10.006691 (+ 6416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:56171: BlockingRecv error: recv got EOF from 127.7.244.65:56171 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 12 00:35:10 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:10 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:10.379106 27010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:10.369461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:37253)
0812 00:35:10.370059 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:10.370077 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:10.370222 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:10.370596 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:10.370601 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:10.371092 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:10.371253 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:10.371262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.372342 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.372346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:10.373422 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:10.373437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.374327 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.374334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:10.374506 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:10.375178 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:10.375198 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:10.378747 (+ 3549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":338,"threads_started":1}
W20260812 00:35:10.379302 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:10.369625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:37253 (local address 127.7.244.126:45309)
0812 00:35:10.369862 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:10.369870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:10.370165 (+ 295us) server_negotiation.cc:407] Connection header received
0812 00:35:10.370379 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:10.370383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:10.370451 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:10.370536 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:10.371415 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.372185 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.373573 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.374118 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.374243 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:10.379051 (+ 4808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:37253: BlockingRecv error: recv got EOF from 127.7.244.66:37253 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 00:35:10 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:10 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:10.782102 27011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:10.772529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:54841)
0812 00:35:10.773086 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:10.773104 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:10.773229 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:10.773564 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:10.773567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:10.774053 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:10.774213 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:10.774223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.775440 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.775445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:10.776475 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:10.776492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.777316 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.777321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:10.777497 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:10.778158 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:10.778178 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:10.781797 (+ 3619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":311,"threads_started":1}
W20260812 00:35:10.782389 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:10.772640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:54841 (local address 127.7.244.126:45309)
0812 00:35:10.772822 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:10.772827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:10.773170 (+ 343us) server_negotiation.cc:407] Connection header received
0812 00:35:10.773355 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:10.773359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:10.773440 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:10.773536 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:10.774382 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.775260 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.776650 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:10.777102 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:10.777220 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:10.782133 (+ 4913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:54841: BlockingRecv error: recv got EOF from 127.7.244.67:54841 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 00:35:11 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:11 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:11.018018 27018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:11.007963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:41893)
0812 00:35:11.008596 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:11.008614 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:11.008796 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:11.009161 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:11.009166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:11.009654 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:11.009880 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:11.009892 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.011064 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.011070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:11.012276 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:11.012296 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.013057 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.013063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:11.013237 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:11.013875 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:11.013895 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:11.017653 (+ 3758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":368,"threads_started":1}
W20260812 00:35:11.018234 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:11.008174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:41893 (local address 127.7.244.126:45309)
0812 00:35:11.008441 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:11.008448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:11.008722 (+ 274us) server_negotiation.cc:407] Connection header received
0812 00:35:11.008938 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:11.008942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:11.009010 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:11.009097 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:11.010059 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.010889 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.012426 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.012858 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.013000 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:11.017984 (+ 4984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:41893: BlockingRecv error: recv got EOF from 127.7.244.65:41893 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 12 00:35:11 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:11 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:11.390070 27019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:11.380449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:33785)
0812 00:35:11.380957 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:11.380974 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:11.381099 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:11.381421 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:11.381424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:11.381830 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:11.382025 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:11.382035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.383061 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.383065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:11.384052 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:11.384068 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.384784 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.384788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:11.384918 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:11.385778 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:11.385799 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:11.389804 (+ 4005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":288,"threads_started":1}
W20260812 00:35:11.390259 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:11.380541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:33785 (local address 127.7.244.126:45309)
0812 00:35:11.380742 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:11.380746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:11.381038 (+ 292us) server_negotiation.cc:407] Connection header received
0812 00:35:11.381196 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:11.381199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:11.381265 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:11.381363 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:11.382144 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.382919 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.384205 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.384649 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.384749 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:11.390051 (+ 5302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:33785: BlockingRecv error: recv got EOF from 127.7.244.66:33785 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 00:35:11 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:11 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:11.792546 27020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:11.783209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:57059)
0812 00:35:11.783768 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:11.783788 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:11.783927 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:11.784272 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:11.784276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:11.784718 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:11.784935 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:11.784946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.786050 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.786055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:11.787085 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:11.787101 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.787787 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.787791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:11.787920 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:11.788563 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:11.788582 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:11.792261 (+ 3679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":307,"threads_started":1}
W20260812 00:35:11.792757 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:11.783313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:57059 (local address 127.7.244.126:45309)
0812 00:35:11.783476 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:11.783480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:11.783860 (+ 380us) server_negotiation.cc:407] Connection header received
0812 00:35:11.784030 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:11.784033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:11.784112 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:11.784195 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:11.785081 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.785889 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.787234 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:11.787646 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:11.787754 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:11.792527 (+ 4773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:57059: BlockingRecv error: recv got EOF from 127.7.244.67:57059 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 00:35:12 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:12 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:12.029042 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:12.019391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:58735)
0812 00:35:12.019975 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:12.019993 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:12.020146 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:12.020540 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:12.020545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:12.020961 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:12.021164 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:12.021175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.022557 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.022563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:12.023513 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:12.023530 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.024286 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.024293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:12.024480 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:12.025100 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:12.025120 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:12.028693 (+ 3573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":327,"threads_started":1}
W20260812 00:35:12.029232 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:12.019553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:58735 (local address 127.7.244.126:45309)
0812 00:35:12.019782 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:12.019789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:12.020088 (+ 299us) server_negotiation.cc:407] Connection header received
0812 00:35:12.020313 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:12.020317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:12.020386 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:12.020474 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:12.021358 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.022330 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.023673 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.024115 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.024233 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:12.028981 (+ 4748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:58735: BlockingRecv error: recv got EOF from 127.7.244.65:58735 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 12 00:35:12 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:12 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:12.400559 27024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:12.391205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:56755)
0812 00:35:12.391746 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:12.391763 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:12.391894 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:12.392216 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:12.392219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:12.392595 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:12.392775 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:12.392785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.393842 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.393846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:12.394780 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:12.394793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.395475 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.395480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:12.395644 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:12.396458 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:12.396483 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:12.400286 (+ 3803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":290,"threads_started":1}
W20260812 00:35:12.400769 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:12.391366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:56755 (local address 127.7.244.126:45309)
0812 00:35:12.391624 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:12.391630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:12.391878 (+ 248us) server_negotiation.cc:407] Connection header received
0812 00:35:12.391997 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:12.392001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:12.392089 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:12.392184 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:12.392920 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.393721 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.394897 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.395302 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.395393 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:12.400535 (+ 5142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:56755: BlockingRecv error: recv got EOF from 127.7.244.66:56755 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 12 00:35:12 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:12 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:12.803124 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:12.793786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:45043)
0812 00:35:12.794316 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:12.794336 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:12.794469 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:12.794848 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:12.794852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:12.795277 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:12.795465 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:12.795475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.796535 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.796540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:12.797533 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:12.797548 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.798244 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.798248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:12.798382 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:12.798995 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:12.799012 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:12.802790 (+ 3778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":277,"threads_started":1}
W20260812 00:35:12.803260 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:12.793965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:45043 (local address 127.7.244.126:45309)
0812 00:35:12.794232 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:12.794238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:12.794418 (+ 180us) server_negotiation.cc:407] Connection header received
0812 00:35:12.794580 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:12.794584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:12.794669 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:12.794756 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:12.795591 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.796345 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.797658 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:12.798118 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:12.798233 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:12.803040 (+ 4807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:45043: BlockingRecv error: recv got EOF from 127.7.244.67:45043 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 12 00:35:13 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:13 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:13.039821 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:13.030142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:56983)
0812 00:35:13.030749 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:13.030768 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:13.030893 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:13.031174 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:13.031177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:13.031618 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:13.031805 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:13.031815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.032894 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.032899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:13.033994 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:13.034011 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.034822 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.034828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:13.034975 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:13.035638 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:13.035658 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:13.039454 (+ 3796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":361,"threads_started":1}
W20260812 00:35:13.040032 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:13.030232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:56983 (local address 127.7.244.126:45309)
0812 00:35:13.030426 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:13.030431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:13.030836 (+ 405us) server_negotiation.cc:407] Connection header received
0812 00:35:13.030988 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:13.030992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:13.031057 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:13.031139 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:13.031963 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.032722 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.034183 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.034649 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.034758 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:13.039751 (+ 4993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:56983: BlockingRecv error: recv got EOF from 127.7.244.65:56983 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 00:35:13 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:13 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:13.411935 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:13.401831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:51367)
0812 00:35:13.402494 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:13.402511 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:13.402664 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:13.403090 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:13.403096 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:13.403517 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:13.403692 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:13.403702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.404783 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.404789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:13.406245 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:13.406266 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.407303 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.407310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:13.407507 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:13.408112 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:13.408128 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:13.411648 (+ 3520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":394,"threads_started":1}
W20260812 00:35:13.412191 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:13.402026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:51367 (local address 127.7.244.126:45309)
0812 00:35:13.402303 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:13.402310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:13.402607 (+ 297us) server_negotiation.cc:407] Connection header received
0812 00:35:13.402808 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:13.402813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:13.402920 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:13.403010 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:13.403857 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.404624 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.406405 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.407102 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.407238 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:13.411937 (+ 4699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:51367: BlockingRecv error: recv got EOF from 127.7.244.66:51367 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 12 00:35:13 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:13 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:13.814965 27031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:13.804453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:47367)
0812 00:35:13.805042 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:13.805059 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:13.805198 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:13.805619 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:13.805624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:13.806037 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:13.806218 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:13.806229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.807347 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.807351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:13.808665 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:13.808683 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.809692 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.809701 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:13.810214 (+ 513us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:13.811077 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:13.811103 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:13.814610 (+ 3507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":333,"threads_started":1}
W20260812 00:35:13.815146 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:13.804567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:47367 (local address 127.7.244.126:45309)
0812 00:35:13.804854 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:13.804861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:13.805150 (+ 289us) server_negotiation.cc:407] Connection header received
0812 00:35:13.805367 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:13.805373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:13.805447 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:13.805534 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:13.806442 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.807201 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.808849 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:13.809464 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:13.809602 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:13.814855 (+ 5253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:47367: BlockingRecv error: recv got EOF from 127.7.244.67:47367 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Aug 12 00:35:14 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:14 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:14.050107 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:14.040967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:45097)
0812 00:35:14.041534 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:14.041551 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:14.041692 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:14.042026 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:14.042031 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:14.042533 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:14.042694 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:14.042704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.043745 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.043750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:14.044692 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:14.044705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.045449 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.045455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:14.045652 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:14.046240 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:14.046259 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:14.049753 (+ 3494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":308,"threads_started":1}
W20260812 00:35:14.050267 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:14.041100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:45097 (local address 127.7.244.126:45309)
0812 00:35:14.041340 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:14.041346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:14.041643 (+ 297us) server_negotiation.cc:407] Connection header received
0812 00:35:14.041811 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:14.041816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:14.041884 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:14.041967 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:14.042854 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.043594 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.044843 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.045272 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.045435 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:14.050051 (+ 4616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:45097: BlockingRecv error: recv got EOF from 127.7.244.65:45097 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 00:35:14 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:14 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:14.422993 27038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:14.412990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:60147)
0812 00:35:14.413571 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:14.413591 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:14.413738 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:14.414108 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:14.414114 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:14.414579 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:14.414752 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:14.414763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.415866 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.415872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:14.417029 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:14.417045 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.417923 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.417929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:14.418132 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:14.418894 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:14.418915 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:14.422705 (+ 3790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":245,"threads_started":1}
W20260812 00:35:14.423262 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:14.413328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:60147 (local address 127.7.244.126:45309)
0812 00:35:14.413544 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:14.413550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:14.413670 (+ 120us) server_negotiation.cc:407] Connection header received
0812 00:35:14.413862 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:14.413867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:14.413954 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:14.414043 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:14.414921 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.415718 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.417186 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.417761 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.417880 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:14.422993 (+ 5113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:60147: BlockingRecv error: recv got EOF from 127.7.244.66:60147 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 00:35:14 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:14 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:14.829221 27039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:14.816233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:33009)
0812 00:35:14.816820 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:14.816837 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:14.816965 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:14.817319 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:14.817324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:14.817825 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:14.817995 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:14.818005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.819064 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.819068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:14.819996 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:14.820009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.820879 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.820884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:14.821035 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:14.824458 (+ 3423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:14.824482 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:14.828887 (+ 4405us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":328,"threads_started":1}
W20260812 00:35:14.829423 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:14.816359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:33009 (local address 127.7.244.126:45309)
0812 00:35:14.816539 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:14.816543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:14.816950 (+ 407us) server_negotiation.cc:407] Connection header received
0812 00:35:14.817076 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:14.817080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:14.817149 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:14.817251 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:14.818150 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.818930 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.820170 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:14.820675 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:14.820797 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:14.829143 (+ 8346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:33009: BlockingRecv error: recv got EOF from 127.7.244.67:33009 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 00:35:15 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:15 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:15.061004 27040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:15.051450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:39061)
0812 00:35:15.051991 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:15.052008 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:15.052143 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:15.052551 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:15.052555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:15.052978 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:15.053159 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:15.053168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.054272 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.054277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:15.055249 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:15.055263 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.056202 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.056208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:15.056423 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:15.057137 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:15.057162 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:15.060761 (+ 3599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":306,"threads_started":1}
W20260812 00:35:15.061234 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:15.051550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:39061 (local address 127.7.244.126:45309)
0812 00:35:15.051790 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:15.051797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:15.052096 (+ 299us) server_negotiation.cc:407] Connection header received
0812 00:35:15.052286 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:15.052290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:15.052362 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:15.052451 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:15.053335 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.054110 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.055444 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.056042 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.056176 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:15.061002 (+ 4826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:39061: BlockingRecv error: recv got EOF from 127.7.244.65:39061 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 00:35:15 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:15 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:15.433823 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:15.424094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:40991)
0812 00:35:15.424667 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:15.424691 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:15.424860 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:15.425318 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:15.425323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:15.425864 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:15.426140 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:15.426155 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.427415 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.427421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:15.428358 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:15.428372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.429109 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.429114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:15.429260 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:15.429923 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:15.429942 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:15.433576 (+ 3634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":308,"threads_started":1}
W20260812 00:35:15.434052 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:15.424271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:40991 (local address 127.7.244.126:45309)
0812 00:35:15.424516 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:15.424523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:15.424792 (+ 269us) server_negotiation.cc:407] Connection header received
0812 00:35:15.425015 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:15.425021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:15.425123 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:15.425237 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:15.426316 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.427266 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.428495 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.428961 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.429054 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:15.433828 (+ 4774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:40991: BlockingRecv error: recv got EOF from 127.7.244.66:40991 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 00:35:15 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:15 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:15.843696 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:15.832589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:44173)
0812 00:35:15.833055 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:15.833077 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:15.833245 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:15.833723 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:15.833728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:15.834285 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:15.834516 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:15.834529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.836006 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.836012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:15.837515 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:15.837538 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.838499 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.838506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:15.838686 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:15.839493 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:15.839517 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:15.843349 (+ 3832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":343,"threads_started":1}
W20260812 00:35:15.844012 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:15.832700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:44173 (local address 127.7.244.126:45309)
0812 00:35:15.832935 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:15.832943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:15.833173 (+ 230us) server_negotiation.cc:407] Connection header received
0812 00:35:15.833432 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:15.833438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:15.833538 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:15.833660 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:15.834673 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.835749 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.837679 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:15.838313 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:15.838439 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:15.843744 (+ 5305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:44173: BlockingRecv error: recv got EOF from 127.7.244.67:44173 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 00:35:16 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:16 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:16.071695 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:16.061968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:56899)
0812 00:35:16.062601 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:16.062618 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:16.062766 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:16.063153 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:16.063158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:16.063627 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:16.063810 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:16.063821 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.064947 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.064952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:16.065952 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:16.065968 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.066771 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.066778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:16.066960 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:16.067576 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:16.067593 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:16.071332 (+ 3739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":295,"threads_started":1}
W20260812 00:35:16.071910 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:16.062108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:56899 (local address 127.7.244.126:45309)
0812 00:35:16.062327 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:16.062333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:16.062712 (+ 379us) server_negotiation.cc:407] Connection header received
0812 00:35:16.062938 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:16.062943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:16.063008 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:16.063090 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:16.064008 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.064789 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.066114 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.066598 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.066721 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:16.071677 (+ 4956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:56899: BlockingRecv error: recv got EOF from 127.7.244.65:56899 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 12 00:35:16 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:16 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:16.445180 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:16.434977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:41025)
0812 00:35:16.435657 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:16.435675 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:16.435834 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:16.436179 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:16.436184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:16.436710 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:16.436901 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:16.436912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.438077 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.438081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:16.439185 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:16.439210 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.440080 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.440086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:16.440228 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:16.440973 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:16.440994 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:16.444922 (+ 3928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":315,"threads_started":1}
W20260812 00:35:16.445480 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:16.435095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:41025 (local address 127.7.244.126:45309)
0812 00:35:16.435404 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:16.435410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:16.435778 (+ 368us) server_negotiation.cc:407] Connection header received
0812 00:35:16.435975 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:16.435979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:16.436046 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:16.436130 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:16.437068 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.437921 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.439345 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.439895 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.440017 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:16.445166 (+ 5149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:41025: BlockingRecv error: recv got EOF from 127.7.244.66:41025 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
Aug 12 00:35:16 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:16 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:16.854332 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:16.844818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:52501)
0812 00:35:16.845476 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:16.845511 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:16.845663 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:16.846144 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:16.846147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:16.846601 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:16.846779 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:16.846789 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.847860 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.847864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:16.848826 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:16.848839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.849599 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.849604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:16.849730 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:16.850374 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:16.850399 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:16.853982 (+ 3583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":259,"threads_started":1}
W20260812 00:35:16.854480 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:16.844855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:52501 (local address 127.7.244.126:45309)
0812 00:35:16.845069 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:16.845076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:16.845599 (+ 523us) server_negotiation.cc:407] Connection header received
0812 00:35:16.845833 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:16.845839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:16.845907 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:16.845963 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:16.846928 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.847706 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.848947 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:16.849450 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:16.849545 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:16.854240 (+ 4695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:52501: BlockingRecv error: recv got EOF from 127.7.244.67:52501 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 00:35:17 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:17 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:17.083561 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:17.072930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:54991)
0812 00:35:17.073553 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:17.073570 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:17.073698 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:17.074150 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:17.074154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:17.074752 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:17.074975 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:17.074989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.076343 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.076348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:17.077365 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:17.077381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.078186 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.078191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:17.078324 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:17.079070 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:17.079092 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:17.083199 (+ 4107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":296,"threads_started":1}
W20260812 00:35:17.083709 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:17.073110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:54991 (local address 127.7.244.126:45309)
0812 00:35:17.073280 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:17.073310 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:17.073651 (+ 341us) server_negotiation.cc:407] Connection header received
0812 00:35:17.073856 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:17.073860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:17.073947 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:17.074041 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:17.075166 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.076148 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.077536 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.078012 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.078118 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:17.083452 (+ 5334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:54991: BlockingRecv error: recv got EOF from 127.7.244.65:54991 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 00:35:17 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:17 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:17.456511 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:17.446373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:37667)
0812 00:35:17.446984 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:17.447001 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:17.447153 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:17.447511 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:17.447516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:17.447960 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:17.448139 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:17.448149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.449479 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.449484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:17.450518 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:17.450533 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.451341 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.451348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:17.451519 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:17.452158 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:17.452176 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:17.456233 (+ 4057us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":325,"threads_started":1}
W20260812 00:35:17.456759 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:17.446561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:37667 (local address 127.7.244.126:45309)
0812 00:35:17.446805 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:17.446811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:17.447098 (+ 287us) server_negotiation.cc:407] Connection header received
0812 00:35:17.447282 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:17.447288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:17.447362 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:17.447457 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:17.448300 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.449278 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.450671 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.451142 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.451280 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:17.456520 (+ 5240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:37667: BlockingRecv error: recv got EOF from 127.7.244.66:37667 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 12 00:35:17 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:17 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:17.865618 27055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:17.855578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:38835)
0812 00:35:17.856142 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:17.856160 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:17.856299 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:17.856656 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:17.856660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:17.857069 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:17.857259 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:17.857273 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.858434 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.858439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:17.859537 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:17.859556 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.860512 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.860519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:17.860695 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:17.861438 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:17.861457 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:17.865327 (+ 3870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":317,"threads_started":1}
W20260812 00:35:17.865839 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:17.855748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:38835 (local address 127.7.244.126:45309)
0812 00:35:17.856007 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:17.856014 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:17.856248 (+ 234us) server_negotiation.cc:407] Connection header received
0812 00:35:17.856452 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:17.856456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:17.856525 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:17.856617 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:17.857482 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.858235 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.859739 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:17.860289 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:17.860405 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:17.865552 (+ 5147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:38835: BlockingRecv error: recv got EOF from 127.7.244.67:38835 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 12 00:35:18 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:18 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:18.095199 27058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:18.085023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:40421)
0812 00:35:18.085683 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:18.085700 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:18.085860 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:18.086209 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:18.086214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:18.086688 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:18.086866 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:18.086876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.087966 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.087971 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:18.089425 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:18.089448 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.090447 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.090452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:18.090593 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:18.091178 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:18.091196 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:18.094870 (+ 3674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":393,"threads_started":1}
W20260812 00:35:18.095535 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:18.085229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:40421 (local address 127.7.244.126:45309)
0812 00:35:18.085534 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:18.085540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:18.085800 (+ 260us) server_negotiation.cc:407] Connection header received
0812 00:35:18.085981 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:18.085987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:18.086058 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:18.086145 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:18.087030 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.087796 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.089626 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.090280 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.090381 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:18.095149 (+ 4768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:40421: BlockingRecv error: recv got EOF from 127.7.244.65:40421 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 12 00:35:18 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:18 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:18.467303 27059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:18.457684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:55539)
0812 00:35:18.458211 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:18.458228 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:18.458369 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:18.458703 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:18.458709 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:18.459172 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:18.459329 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:18.459339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.460431 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.460435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:18.461447 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:18.461461 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.462306 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.462312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:18.462502 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:18.463203 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:18.463221 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:18.467058 (+ 3837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":284,"threads_started":1}
W20260812 00:35:18.467639 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:18.457734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:55539 (local address 127.7.244.126:45309)
0812 00:35:18.457960 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:18.457966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:18.458320 (+ 354us) server_negotiation.cc:407] Connection header received
0812 00:35:18.458497 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:18.458502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:18.458566 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:18.458649 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:18.459482 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.460273 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.461600 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.462109 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.462259 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:18.467359 (+ 5100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:55539: BlockingRecv error: recv got EOF from 127.7.244.66:55539 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 00:35:18 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:18 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:18.876883 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:18.866707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:36281)
0812 00:35:18.867267 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:18.867285 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:18.867438 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:18.867847 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:18.867852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:18.868310 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:18.868490 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:18.868501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.869610 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.869615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:18.870844 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:18.870864 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.871894 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.871899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:18.872049 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:18.872714 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:18.872741 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:18.876621 (+ 3880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":286,"threads_started":1}
W20260812 00:35:18.877152 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:18.866874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:36281 (local address 127.7.244.126:45309)
0812 00:35:18.867122 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:18.867129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:18.867384 (+ 255us) server_negotiation.cc:407] Connection header received
0812 00:35:18.867600 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:18.867604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:18.867679 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:18.867768 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:18.868657 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.869459 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.871021 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:18.871716 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:18.871853 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:18.876904 (+ 5051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:36281: BlockingRecv error: recv got EOF from 127.7.244.67:36281 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 12 00:35:19 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:19 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:19.107154 27062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:19.096389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:44857)
0812 00:35:19.096992 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:19.097010 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:19.097182 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:19.097674 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:19.097680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:19.098056 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:19.098248 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:19.098259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.099837 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.099842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:19.100879 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:19.100897 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.102034 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.102040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:19.102191 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:19.102903 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:19.102923 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:19.106897 (+ 3974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":400,"threads_started":1}
W20260812 00:35:19.107371 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:19.096583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:44857 (local address 127.7.244.126:45309)
0812 00:35:19.096790 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:19.096794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:19.097129 (+ 335us) server_negotiation.cc:407] Connection header received
0812 00:35:19.097393 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:19.097400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:19.097491 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:19.097607 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:19.098420 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.099621 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.101034 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.101810 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.101956 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:19.107113 (+ 5157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:44857: BlockingRecv error: recv got EOF from 127.7.244.65:44857 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 00:35:19 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:19 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:19.477921 27063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:19.468410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:51303)
0812 00:35:19.468986 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:19.469003 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:19.469123 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:19.469446 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:19.469449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:19.469870 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:19.470110 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:19.470123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.471158 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.471162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:19.472220 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:19.472236 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.472949 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.472954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:19.473097 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:19.473830 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:19.473852 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:19.477600 (+ 3748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":316,"threads_started":1}
W20260812 00:35:19.478060 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:19.468435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:51303 (local address 127.7.244.126:45309)
0812 00:35:19.468621 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:19.468626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:19.469084 (+ 458us) server_negotiation.cc:407] Connection header received
0812 00:35:19.469229 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:19.469234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:19.469332 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:19.469423 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:19.470289 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.471032 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.472359 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.472763 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.472859 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:19.477856 (+ 4997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:51303: BlockingRecv error: recv got EOF from 127.7.244.66:51303 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 00:35:19 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:19 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:19.893123 27064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:19.877964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:59309)
0812 00:35:19.878596 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:19.878616 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:19.878782 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:19.879181 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:19.879187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:19.879716 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:19.879903 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:19.879913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.881078 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.881084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:19.882710 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:19.882734 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.883993 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.883998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:19.884177 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:19.888983 (+ 4806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:19.889008 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:19.892771 (+ 3763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":348,"threads_started":1}
W20260812 00:35:19.893380 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:19.878173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:59309 (local address 127.7.244.126:45309)
0812 00:35:19.878445 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:19.878451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:19.878724 (+ 273us) server_negotiation.cc:407] Connection header received
0812 00:35:19.878924 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:19.878928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:19.879026 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:19.879112 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:19.880087 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.880851 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.882910 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:19.883591 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:19.883754 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:19.893062 (+ 9308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:59309: BlockingRecv error: recv got EOF from 127.7.244.67:59309 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Aug 12 00:35:20 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:20 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:20.118544 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:20.108222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:37743)
0812 00:35:20.108811 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:20.108830 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:20.108989 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:20.109419 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:20.109425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:20.109938 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:20.110148 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:20.110161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.111287 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.111294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:20.112305 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:20.112320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.113163 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.113171 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:20.113421 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:20.114160 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:20.114187 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:20.118215 (+ 4028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":318,"threads_started":1}
W20260812 00:35:20.118669 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:20.108377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:37743 (local address 127.7.244.126:45309)
0812 00:35:20.108674 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:20.108681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:20.108923 (+ 242us) server_negotiation.cc:407] Connection header received
0812 00:35:20.109120 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:20.109125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:20.109231 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:20.109373 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:20.110325 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.111108 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.112480 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.112983 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.113114 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:20.118454 (+ 5340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:37743: BlockingRecv error: recv got EOF from 127.7.244.65:37743 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 12 00:35:20 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:20 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:20.488593 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:20.479183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:34907)
0812 00:35:20.479793 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:20.479813 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:20.479953 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:20.480273 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:20.480276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:20.480742 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:20.480935 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:20.480944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.482110 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.482114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:20.483087 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:20.483103 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.483894 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.483900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:20.484050 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:20.484706 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:20.484724 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:20.488343 (+ 3619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":373,"threads_started":1}
W20260812 00:35:20.488814 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:20.479375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:34907 (local address 127.7.244.126:45309)
0812 00:35:20.479549 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:20.479553 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:20.479913 (+ 360us) server_negotiation.cc:407] Connection header received
0812 00:35:20.480060 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:20.480064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:20.480134 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:20.480237 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:20.481101 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.481945 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.483286 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.483703 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.483805 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:20.488563 (+ 4758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:34907: BlockingRecv error: recv got EOF from 127.7.244.66:34907 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 00:35:20 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:20 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:20.904148 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:20.894212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:55579)
0812 00:35:20.894808 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:20.894824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:20.894971 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:20.895388 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:20.895393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:20.895904 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:20.896144 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:20.896156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.897321 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.897328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:20.898294 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:20.898309 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.899108 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.899114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:20.899285 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:20.899920 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:20.899940 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:20.903883 (+ 3943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":357,"threads_started":1}
W20260812 00:35:20.904547 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:20.894383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:55579 (local address 127.7.244.126:45309)
0812 00:35:20.894632 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:20.894638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:20.894920 (+ 282us) server_negotiation.cc:407] Connection header received
0812 00:35:20.895108 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:20.895113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:20.895193 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:20.895317 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:20.896363 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.897125 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.898467 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:20.898936 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:20.899052 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:20.904219 (+ 5167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:55579: BlockingRecv error: recv got EOF from 127.7.244.67:55579 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 12 00:35:21 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:21 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:21.129258 27075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:21.119628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:38267)
0812 00:35:21.120182 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:21.120204 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:21.120347 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:21.120701 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:21.120706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:21.121214 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:21.121444 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:21.121456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.122516 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.122523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:21.123491 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:21.123505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.124349 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.124356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:21.124556 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:21.125186 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:21.125204 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:21.128955 (+ 3751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":290,"threads_started":1}
W20260812 00:35:21.129503 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:21.119669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:38267 (local address 127.7.244.126:45309)
0812 00:35:21.119867 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:21.119873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:21.120275 (+ 402us) server_negotiation.cc:407] Connection header received
0812 00:35:21.120486 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:21.120492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:21.120558 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:21.120635 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:21.121607 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.122359 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.123648 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.124145 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.124295 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:21.129205 (+ 4910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:38267: BlockingRecv error: recv got EOF from 127.7.244.65:38267 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 00:35:21 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:21 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:21.499217 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:21.489653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:35375)
0812 00:35:21.490221 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:21.490238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:21.490359 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:21.490669 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:21.490672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:21.491098 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:21.491279 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:21.491289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.492405 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.492410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:21.493465 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:21.493481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.494329 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.494335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:21.494485 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:21.495125 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:21.495144 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:21.498906 (+ 3762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":303,"threads_started":1}
W20260812 00:35:21.499402 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:21.489772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:35375 (local address 127.7.244.126:45309)
0812 00:35:21.489959 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:21.489963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:21.490313 (+ 350us) server_negotiation.cc:407] Connection header received
0812 00:35:21.490479 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:21.490483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:21.490548 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:21.490635 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:21.491444 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.492227 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.493630 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.494083 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.494194 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:21.499126 (+ 4932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:35375: BlockingRecv error: recv got EOF from 127.7.244.66:35375 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 12 00:35:21 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:21 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:21.914549 27079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:21.905277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:46629)
0812 00:35:21.906033 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:21.906050 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:21.906180 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:21.906528 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:21.906533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:21.906989 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:21.907165 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:21.907175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.908205 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.908210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:21.909147 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:21.909160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.909972 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.909979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:21.910144 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:21.910796 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:21.910820 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:21.914287 (+ 3467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":475,"threads_started":1}
W20260812 00:35:21.914812 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:21.905374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:46629 (local address 127.7.244.126:45309)
0812 00:35:21.905567 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:21.905573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:21.906144 (+ 571us) server_negotiation.cc:407] Connection header received
0812 00:35:21.906311 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:21.906317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:21.906398 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:21.906483 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:21.907317 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.908065 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.909342 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:21.909776 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:21.909889 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:21.914521 (+ 4632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:46629: BlockingRecv error: recv got EOF from 127.7.244.67:46629 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 00:35:22 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:22 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:22.140537 27080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:22.130502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:44591)
0812 00:35:22.131023 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:22.131042 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:22.131192 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:22.131616 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:22.131620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:22.132053 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:22.132215 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:22.132224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.133371 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.133375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:22.134534 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:22.134550 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.135409 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.135415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:22.135562 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:22.136326 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:22.136346 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:22.140272 (+ 3926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":247,"threads_started":1}
W20260812 00:35:22.140827 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:22.130673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:44591 (local address 127.7.244.126:45309)
0812 00:35:22.130890 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:22.130896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:22.131164 (+ 268us) server_negotiation.cc:407] Connection header received
0812 00:35:22.131373 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:22.131377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:22.131451 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:22.131547 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:22.132346 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.133151 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.134681 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.135196 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.135334 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:22.140572 (+ 5238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:44591: BlockingRecv error: recv got EOF from 127.7.244.65:44591 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 00:35:22 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:22 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:22.510612 27081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:22.500637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:60363)
0812 00:35:22.501141 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:22.501158 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:22.501314 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:22.501625 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:22.501628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:22.502079 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:22.502244 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:22.502255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.503939 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.503944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:22.505061 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:22.505080 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.505854 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.505859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:22.505994 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:22.506630 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:22.506650 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:22.510371 (+ 3721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":253,"threads_started":1}
W20260812 00:35:22.510929 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:22.500951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:60363 (local address 127.7.244.126:45309)
0812 00:35:22.501197 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:22.501202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:22.501222 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:35:22.501433 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:22.501437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:22.501503 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:22.501590 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:22.502618 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.503734 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.505236 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.505713 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.505815 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:22.510645 (+ 4830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:60363: BlockingRecv error: recv got EOF from 127.7.244.66:60363 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Aug 12 00:35:22 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:22 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.67@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:22.925575 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:22.915817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.67:51781)
0812 00:35:22.916488 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:22.916505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:22.916679 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:22.917062 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:22.917068 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:22.917560 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:22.917728 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:22.917739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.918836 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.918841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:22.919774 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:22.919788 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.920515 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.920522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:22.920693 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:22.921414 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:22.921434 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:22.925320 (+ 3886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":340,"threads_started":1}
W20260812 00:35:22.925789 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:22.916107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:51781 (local address 127.7.244.126:45309)
0812 00:35:22.916355 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:22.916361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:22.916601 (+ 240us) server_negotiation.cc:407] Connection header received
0812 00:35:22.916825 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:22.916829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:22.916910 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:22.916999 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:22.917895 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.918672 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.919933 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:22.920363 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:22.920476 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:22.925553 (+ 5077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.67:51781: BlockingRecv error: recv got EOF from 127.7.244.67:51781 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 12 00:35:23 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:23 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.65@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:23.151475 27085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:23.141809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.65:41253)
0812 00:35:23.142331 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:23.142349 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:23.142471 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:23.142759 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:23.142762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:23.143146 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:23.143370 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:23.143382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.144531 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.144537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:23.145606 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:23.145624 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.146454 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.146460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:23.146634 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:23.147238 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:23.147257 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:23.151072 (+ 3815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":240,"threads_started":1}
W20260812 00:35:23.151741 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:23.141929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:41253 (local address 127.7.244.126:45309)
0812 00:35:23.142160 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:23.142165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:23.142454 (+ 289us) server_negotiation.cc:407] Connection header received
0812 00:35:23.142569 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:23.142573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:23.142642 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:23.142729 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:23.143552 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.144325 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.145791 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.146253 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.146364 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:23.151411 (+ 5047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.65:41253: BlockingRecv error: recv got EOF from 127.7.244.65:41253 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 12 00:35:23 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 00:35:23 dist-test-slave-gjw7 krb5kdc[24869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.244.66@KRBTEST.COM for oryx/127.7.244.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 00:35:23.521526 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:23.511991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:45309 (local address 127.7.244.66:40255)
0812 00:35:23.512592 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:23.512611 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:23.512765 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:23.513138 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:23.513143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:23.513639 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:23.513843 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:23.513854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.514960 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.514966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:23.515904 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:23.515933 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.516668 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.516674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:23.516850 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:23.517557 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:23.517577 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:23.521227 (+ 3650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.244.126:45309: Server oryx/127.7.244.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":268,"threads_started":1}
W20260812 00:35:23.521772 27004 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:23.512191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:40255 (local address 127.7.244.126:45309)
0812 00:35:23.512443 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:23.512450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:23.512705 (+ 255us) server_negotiation.cc:407] Connection header received
0812 00:35:23.512903 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:23.512908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:23.512975 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:23.513064 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:23.514014 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.514789 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.516077 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.516505 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.516621 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:23.521543 (+ 4922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.244.66:40255: BlockingRecv error: recv got EOF from 127.7.244.66:40255 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 00:35:23.696211 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 26112
I20260812 00:35:23.711071 27092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:23.697492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.7.244.65:43017)
0812 00:35:23.698224 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:23.698231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:23.698262 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:35:23.698332 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:23.698353 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:23.698608 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:23.698790 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:23.700394 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.701659 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.703108 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.703733 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.703882 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:23.707180 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:23.707194 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:23.707212 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:23.707273 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:23.710308 (+ 3035us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.244.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 00:35:23.710752 (+ 444us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47554: Request ticket server kudu/127.7.244.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":380,"threads_started":1}
W20260812 00:35:23.711198 27092 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47554: Request ticket server kudu/127.7.244.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 00:35:23.711381 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 26112: Not authorized: Client connection negotiation failed: client connection to 127.7.244.65:43017: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.244.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 00:35:23.711627 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 26112
I20260812 00:35:23.752027 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 26443
I20260812 00:35:23.766563 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:23.753435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.7.244.66:34823)
0812 00:35:23.754189 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:23.754199 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:23.754238 (+ 39us) server_negotiation.cc:407] Connection header received
0812 00:35:23.754321 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:23.754330 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:23.754687 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:23.754950 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:23.756244 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.757440 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.758821 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.759505 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.759639 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:23.762514 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:23.762526 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:23.762543 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:23.762606 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:23.765910 (+ 3304us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.244.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 00:35:23.766281 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40744: Request ticket server kudu/127.7.244.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":327,"threads_started":1}
W20260812 00:35:23.766693 27093 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40744: Request ticket server kudu/127.7.244.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 00:35:23.766850 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 26443: Not authorized: Client connection negotiation failed: client connection to 127.7.244.66:34823: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.244.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 00:35:23.767016 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 26443
I20260812 00:35:23.797027 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 26731
I20260812 00:35:23.810912 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:23.798188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35494 (local address 127.7.244.67:41309)
0812 00:35:23.798798 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:23.798806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:23.798836 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:35:23.798911 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:23.798918 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:23.799158 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:23.799341 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:23.801163 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.802384 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.803636 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:23.804357 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:23.804480 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:23.807233 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:23.807243 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:23.807260 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:23.807309 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:23.810314 (+ 3005us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.244.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 00:35:23.810643 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35494: Request ticket server kudu/127.7.244.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":289,"threads_started":1}
W20260812 00:35:23.811028 27097 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35494: Request ticket server kudu/127.7.244.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 00:35:23.811110 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 26731: Not authorized: Client connection negotiation failed: client connection to 127.7.244.67:41309: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.244.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 00:35:23.811300 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 26731
I20260812 00:35:23.857753 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 25974
I20260812 00:35:23.859201 26103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:23.982685 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 25974
2026-08-12T00:35:24Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22013 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T00:35:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:35:24Z Disabled control of system clock
I20260812 00:35:24.078572 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42265
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:34239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 00:35:24.437973 27105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:24.438292 27105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:24.438400 27105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:24.446007 27105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:35:24.446138 27105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:24.446178 27105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:24.446254 27105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:35:24.446311 27105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:35:24.456327 27105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:24.457841 27105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:24.459774 27105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:24.472477 27118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:24.472519 27114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:24.472440 27126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:24.473558 27105 server_base.cc:1034] running on GCE node
I20260812 00:35:24.474478 27105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:24.476183 27105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:24.477433 27105 hybrid_clock.cc:648] HybridClock initialized: now 1786494924477366 us; error 62 us; skew 500 ppm
I20260812 00:35:24.477849 27105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:35:24.480309 27105 webserver.cc:460] Webserver started at http://127.7.244.126:43215/ using document root <none> and password file <none>
I20260812 00:35:24.481083 27105 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:24.481195 27105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:24.481599 27105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:24.484943 27105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "188b6def5c794c558c8e78321b3661b7"
format_stamp: "Formatted at 2026-08-12 00:35:24 on dist-test-slave-gjw7"
server_key: "b9e8632f67b92042935072a6b492d486"
server_key_iv: "52f1e45efd8f5c645784498bd970dcf3"
server_key_version: "encryptionkey@0"
I20260812 00:35:24.485890 27105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "188b6def5c794c558c8e78321b3661b7"
format_stamp: "Formatted at 2026-08-12 00:35:24 on dist-test-slave-gjw7"
server_key: "b9e8632f67b92042935072a6b492d486"
server_key_iv: "52f1e45efd8f5c645784498bd970dcf3"
server_key_version: "encryptionkey@0"
I20260812 00:35:24.491736 27105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:35:24.495635 27135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:24.497150 27105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:35:24.497381 27105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "188b6def5c794c558c8e78321b3661b7"
format_stamp: "Formatted at 2026-08-12 00:35:24 on dist-test-slave-gjw7"
server_key: "b9e8632f67b92042935072a6b492d486"
server_key_iv: "52f1e45efd8f5c645784498bd970dcf3"
server_key_version: "encryptionkey@0"
I20260812 00:35:24.497632 27105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:24.523972 27105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:24.524972 27105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:24.525312 27105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:24.544621 27105 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:42265
I20260812 00:35:24.544607 27228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:42265 every 8 connection(s)
I20260812 00:35:24.546216 27105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:35:24.548449 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 27105
I20260812 00:35:24.548717 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:35:24.549712 8145 external_mini_cluster.cc:1428] Setting key 93c249054d930a68b97a588c9eb8feac
I20260812 00:35:24.551728 27229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:35:24.566339 27237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:24.552197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.7.244.126:42265)
0812 00:35:24.553284 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:24.553334 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:24.553389 (+ 55us) server_negotiation.cc:407] Connection header received
0812 00:35:24.555100 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:24.555128 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:24.555711 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:24.556173 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:24.556994 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:24.558548 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:24.560636 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:24.561735 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:24.561929 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:24.563477 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:24.563486 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:24.563509 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:24.564175 (+ 666us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:24.564318 (+ 143us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:24.564349 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:24.564465 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:24.564751 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 00:35:24.565143 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":360,"threads_started":1}
I20260812 00:35:24.566998 27229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7: Bootstrap starting.
I20260812 00:35:24.570916 27229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7: Neither blocks nor log segments found. Creating new log.
I20260812 00:35:24.572113 27229 log.cc:824] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7: Log is configured to *not* fsync() on all Append() calls
I20260812 00:35:24.575829 27229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7: No bootstrap required, opened a new log
I20260812 00:35:24.583595 27229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188b6def5c794c558c8e78321b3661b7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42265 } }
I20260812 00:35:24.584090 27229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:35:24.584213 27229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188b6def5c794c558c8e78321b3661b7, State: Initialized, Role: FOLLOWER
I20260812 00:35:24.584846 27229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188b6def5c794c558c8e78321b3661b7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42265 } }
I20260812 00:35:24.585068 27229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:35:24.585203 27229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:35:24.585413 27229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:35:24.587656 27229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188b6def5c794c558c8e78321b3661b7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42265 } }
I20260812 00:35:24.588201 27229 leader_election.cc:302] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 188b6def5c794c558c8e78321b3661b7; no voters:
I20260812 00:35:24.588836 27229 leader_election.cc:288] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:35:24.588996 27240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:35:24.589738 27240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [term 1 LEADER]: Becoming Leader. State: Replica: 188b6def5c794c558c8e78321b3661b7, State: Running, Role: LEADER
I20260812 00:35:24.590291 27240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188b6def5c794c558c8e78321b3661b7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42265 } }
I20260812 00:35:24.591012 27229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:35:24.592958 27244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 188b6def5c794c558c8e78321b3661b7. Latest consensus state: current_term: 1 leader_uuid: "188b6def5c794c558c8e78321b3661b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188b6def5c794c558c8e78321b3661b7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42265 } } }
I20260812 00:35:24.593382 27244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:24.593665 27243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "188b6def5c794c558c8e78321b3661b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188b6def5c794c558c8e78321b3661b7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42265 } } }
I20260812 00:35:24.593986 27243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:24.598332 27248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:35:24.604317 27248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:35:24.614010 27248 catalog_manager.cc:1347] Generated new cluster ID: d635f73f42294bab853b50f14793f193
I20260812 00:35:24.614147 27248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:35:24.630429 27248 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:35:24.631810 27248 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:35:24.645117 27248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 188b6def5c794c558c8e78321b3661b7: Generated new TSK 0
I20260812 00:35:24.646090 27248 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:35:24.654006 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42265
--builtin_ntp_servers=127.7.244.84:34239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 00:35:25.039218 27278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:25.039551 27278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:25.039691 27278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:25.047429 27278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:25.047565 27278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:25.047729 27278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:35:25.057565 27278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:25.058980 27278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:25.060979 27278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:25.078423 27289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:25.079469 27290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:25.083148 27297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:25.083484 27278 server_base.cc:1034] running on GCE node
I20260812 00:35:25.084306 27278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:25.085458 27278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:25.086758 27278 hybrid_clock.cc:648] HybridClock initialized: now 1786494925086692 us; error 66 us; skew 500 ppm
I20260812 00:35:25.087100 27278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:35:25.100176 27278 webserver.cc:460] Webserver started at http://127.7.244.65:44701/ using document root <none> and password file <none>
I20260812 00:35:25.101064 27278 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:25.101236 27278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:25.101653 27278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:25.105434 27278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "27a452afed5748a093b5d5a685c8a5de"
format_stamp: "Formatted at 2026-08-12 00:35:25 on dist-test-slave-gjw7"
server_key: "a4eb0975926faf7e4601c7d2e722f09b"
server_key_iv: "2b290284daf9c7897146ab532c54553e"
server_key_version: "encryptionkey@0"
I20260812 00:35:25.106413 27278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "27a452afed5748a093b5d5a685c8a5de"
format_stamp: "Formatted at 2026-08-12 00:35:25 on dist-test-slave-gjw7"
server_key: "a4eb0975926faf7e4601c7d2e722f09b"
server_key_iv: "2b290284daf9c7897146ab532c54553e"
server_key_version: "encryptionkey@0"
I20260812 00:35:25.114464 27278 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260812 00:35:25.119861 27306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:25.121973 27278 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 00:35:25.122221 27278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "27a452afed5748a093b5d5a685c8a5de"
format_stamp: "Formatted at 2026-08-12 00:35:25 on dist-test-slave-gjw7"
server_key: "a4eb0975926faf7e4601c7d2e722f09b"
server_key_iv: "2b290284daf9c7897146ab532c54553e"
server_key_version: "encryptionkey@0"
I20260812 00:35:25.122532 27278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:25.147418 27278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:25.148655 27278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:25.149040 27278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:25.150418 27278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:25.152617 27278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:25.152755 27278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:25.152878 27278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:25.152958 27278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:25.191361 27278 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:40875
I20260812 00:35:25.191409 27518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:40875 every 8 connection(s)
I20260812 00:35:25.192787 27278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:35:25.194180 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 27278
I20260812 00:35:25.194391 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:35:25.195223 8145 external_mini_cluster.cc:1428] Setting key 8ec1235fb84585546c2bedf8cd08dab1
I20260812 00:35:25.198189 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42265
--builtin_ntp_servers=127.7.244.84:34239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 00:35:25.207891 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:25.195773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:53693 (local address 127.7.244.126:42265)
0812 00:35:25.196251 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:25.196256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:25.197842 (+ 1586us) server_negotiation.cc:407] Connection header received
0812 00:35:25.199968 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:25.199975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:25.200070 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:25.200183 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:25.201567 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:25.202582 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:25.204428 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:25.205008 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:25.205141 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:25.206905 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:25.206912 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:25.206916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:25.206942 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:25.206998 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:25.207016 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:25.207109 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:25.207397 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 00:35:25.207516 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":240,"threads_started":1}
I20260812 00:35:25.209159 27525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:25.196514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42265 (local address 127.7.244.65:53693)
0812 00:35:25.197652 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:25.197712 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:25.199595 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:25.200445 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:25.200455 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:25.200558 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:25.201349 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:25.201376 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:25.202808 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:25.202813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:25.204225 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:25.204251 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:25.205379 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:25.205388 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:25.205637 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:25.206475 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:35:25.206479 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:25.206540 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:35:25.206635 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:25.207153 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:25.207165 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:25.207173 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:35:25.207349 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 00:35:25.207721 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":349,"threads_started":1}
I20260812 00:35:25.211097 27519 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42265
I20260812 00:35:25.211503 27519 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:25.212716 27519 heartbeater.cc:511] Master 127.7.244.126:42265 requested a full tablet report, sending...
I20260812 00:35:25.215334 27169 ts_manager.cc:194] Registered new tserver with Master: 27a452afed5748a093b5d5a685c8a5de (127.7.244.65:40875)
I20260812 00:35:25.217126 27169 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.65:53693
W20260812 00:35:25.557027 27527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:25.557397 27527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:25.557533 27527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:25.565061 27527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:25.565205 27527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:25.565390 27527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:35:25.575304 27527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:25.576764 27527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:25.578876 27527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:25.593570 27537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:25.593639 27544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:25.593530 27541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:25.594282 27527 server_base.cc:1034] running on GCE node
I20260812 00:35:25.594990 27527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:25.596103 27527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:25.597358 27527 hybrid_clock.cc:648] HybridClock initialized: now 1786494925597245 us; error 71 us; skew 500 ppm
I20260812 00:35:25.597697 27527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:35:25.600382 27527 webserver.cc:460] Webserver started at http://127.7.244.66:35645/ using document root <none> and password file <none>
I20260812 00:35:25.601197 27527 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:25.601341 27527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:25.601725 27527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:25.604885 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "d806e38de754454c89071908f3c5a0ad"
format_stamp: "Formatted at 2026-08-12 00:35:25 on dist-test-slave-gjw7"
server_key: "fee5f3f9ddef7c774289577c3de5d0d6"
server_key_iv: "93e9ae2adbc32081e494f3107693afdb"
server_key_version: "encryptionkey@0"
I20260812 00:35:25.605790 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "d806e38de754454c89071908f3c5a0ad"
format_stamp: "Formatted at 2026-08-12 00:35:25 on dist-test-slave-gjw7"
server_key: "fee5f3f9ddef7c774289577c3de5d0d6"
server_key_iv: "93e9ae2adbc32081e494f3107693afdb"
server_key_version: "encryptionkey@0"
I20260812 00:35:25.611757 27527 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260812 00:35:25.615957 27551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:25.617604 27527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:35:25.617805 27527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "d806e38de754454c89071908f3c5a0ad"
format_stamp: "Formatted at 2026-08-12 00:35:25 on dist-test-slave-gjw7"
server_key: "fee5f3f9ddef7c774289577c3de5d0d6"
server_key_iv: "93e9ae2adbc32081e494f3107693afdb"
server_key_version: "encryptionkey@0"
I20260812 00:35:25.618057 27527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:25.644140 27527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:25.645144 27527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:25.645514 27527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:25.646636 27527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:25.648306 27527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:25.648416 27527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:25.648532 27527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:25.648603 27527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:25.681883 27527 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:39495
I20260812 00:35:25.681973 27779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:39495 every 8 connection(s)
I20260812 00:35:25.683326 27527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:35:25.690541 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 27527
I20260812 00:35:25.690729 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:35:25.691159 8145 external_mini_cluster.cc:1428] Setting key d4cfd9d3f7c5565d68a37d5617cffafc
I20260812 00:35:25.693958 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42265
--builtin_ntp_servers=127.7.244.84:34239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 00:35:25.700661 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:25.685967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:37495 (local address 127.7.244.126:42265)
0812 00:35:25.686149 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:25.686153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:25.687656 (+ 1503us) server_negotiation.cc:407] Connection header received
0812 00:35:25.689536 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:25.689543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:25.689629 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:25.689720 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:25.690939 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:25.691774 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:25.693401 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:25.693813 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:25.693948 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:25.699703 (+ 5755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:25.699708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:25.699711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:25.699733 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:25.699771 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:25.699785 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:25.699873 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:25.700227 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 00:35:25.700348 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:35:25.701857 27785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:25.686523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42265 (local address 127.7.244.66:37495)
0812 00:35:25.687482 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:25.687521 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:25.689198 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:25.689984 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:25.689994 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:25.690099 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:25.690749 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:25.690776 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:25.691939 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:25.691946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:25.693236 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:25.693250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:25.694057 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:25.694065 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:25.694333 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:25.699267 (+ 4934us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:35:25.699270 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:25.699323 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:35:25.699447 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:25.699888 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:25.699903 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:25.699911 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:35:25.700159 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 00:35:25.700547 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":297,"threads_started":1}
I20260812 00:35:25.703502 27781 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42265
I20260812 00:35:25.703825 27781 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:25.704625 27781 heartbeater.cc:511] Master 127.7.244.126:42265 requested a full tablet report, sending...
I20260812 00:35:25.706182 27169 ts_manager.cc:194] Registered new tserver with Master: d806e38de754454c89071908f3c5a0ad (127.7.244.66:39495)
I20260812 00:35:25.707005 27169 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.66:37495
W20260812 00:35:26.064428 27786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:26.064736 27786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:26.064858 27786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:26.072474 27786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:26.072592 27786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:26.072736 27786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:35:26.082507 27786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42265
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:26.083910 27786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:26.085932 27786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:26.100068 27805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:26.100103 27795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:26.100136 27798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:26.100936 27786 server_base.cc:1034] running on GCE node
I20260812 00:35:26.101819 27786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:26.102945 27786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:26.104188 27786 hybrid_clock.cc:648] HybridClock initialized: now 1786494926104124 us; error 48 us; skew 500 ppm
I20260812 00:35:26.104559 27786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:35:26.107188 27786 webserver.cc:460] Webserver started at http://127.7.244.67:44157/ using document root <none> and password file <none>
I20260812 00:35:26.108023 27786 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:26.108137 27786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:26.108494 27786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:26.111680 27786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "9f3ed20508894d7b825a1ceb75d66d7f"
format_stamp: "Formatted at 2026-08-12 00:35:26 on dist-test-slave-gjw7"
server_key: "4aeee66145846c003e623fa1bd10cec9"
server_key_iv: "b23e6b6dbc410f638407fd643e5d8a2a"
server_key_version: "encryptionkey@0"
I20260812 00:35:26.112571 27786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "9f3ed20508894d7b825a1ceb75d66d7f"
format_stamp: "Formatted at 2026-08-12 00:35:26 on dist-test-slave-gjw7"
server_key: "4aeee66145846c003e623fa1bd10cec9"
server_key_iv: "b23e6b6dbc410f638407fd643e5d8a2a"
server_key_version: "encryptionkey@0"
I20260812 00:35:26.118420 27786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260812 00:35:26.122495 27815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:26.124274 27786 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:35:26.124472 27786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "9f3ed20508894d7b825a1ceb75d66d7f"
format_stamp: "Formatted at 2026-08-12 00:35:26 on dist-test-slave-gjw7"
server_key: "4aeee66145846c003e623fa1bd10cec9"
server_key_iv: "b23e6b6dbc410f638407fd643e5d8a2a"
server_key_version: "encryptionkey@0"
I20260812 00:35:26.124722 27786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:26.141544 27786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:26.142518 27786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:26.142869 27786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:26.143949 27786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:26.145627 27786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:26.145735 27786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:26.145869 27786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:26.145942 27786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:26.178344 27786 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:33039
I20260812 00:35:26.178423 28060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:33039 every 8 connection(s)
I20260812 00:35:26.179728 27786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:35:26.184151 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 27786
I20260812 00:35:26.184330 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:35:26.184826 8145 external_mini_cluster.cc:1428] Setting key 60c4cc4b6fae462a1448158b973ae4e3
I20260812 00:35:26.194269 27524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:26.182470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:60017 (local address 127.7.244.126:42265)
0812 00:35:26.182727 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:26.182733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:26.184374 (+ 1641us) server_negotiation.cc:407] Connection header received
0812 00:35:26.186590 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:26.186596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:26.186691 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:26.186794 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:26.188466 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.189220 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.190924 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.191572 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.191716 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:26.193385 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:26.193393 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:26.193398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:26.193424 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:26.193471 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:26.193488 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:26.193583 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:26.193790 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 00:35:26.193907 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 00:35:26.195384 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:26.183094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42265 (local address 127.7.244.67:60017)
0812 00:35:26.184102 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:26.184150 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:26.186241 (+ 2091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:26.187080 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:26.187092 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:26.187237 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:26.188245 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:26.188279 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.189422 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.189426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:26.190742 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:26.190756 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.191833 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.191840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:26.192071 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:26.192966 (+ 895us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:35:26.192969 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:26.193015 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:35:26.193110 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:26.193584 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:26.193594 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:26.193601 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 00:35:26.193761 (+ 160us) client_negotiation.cc:240] Negotiation successful
0812 00:35:26.194107 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":297,"threads_started":1}
I20260812 00:35:26.196981 28061 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42265
I20260812 00:35:26.197386 28061 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:26.198308 28061 heartbeater.cc:511] Master 127.7.244.126:42265 requested a full tablet report, sending...
I20260812 00:35:26.200057 27166 ts_manager.cc:194] Registered new tserver with Master: 9f3ed20508894d7b825a1ceb75d66d7f (127.7.244.67:33039)
I20260812 00:35:26.201174 27166 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.67:60017
I20260812 00:35:26.201195 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 00:35:26.207502 27524 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:26.205964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.7.244.126:42265)
0812 00:35:26.206240 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:26.206248 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:26.206623 (+ 375us) server_negotiation.cc:407] Connection header received
0812 00:35:26.206755 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:26.206758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:26.206826 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:26.206910 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:26.207279 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51018: BlockingRecv error: recv got EOF from 127.0.0.1:51018 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 00:35:26.208503 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 27278
I20260812 00:35:26.216727 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:26.209498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44656 (local address 127.7.244.65:40875)
0812 00:35:26.210037 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:26.210044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:26.210072 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:35:26.210137 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:26.210144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:26.210355 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:26.210490 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:26.210893 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.211932 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.213325 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.213973 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.214095 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:26.215270 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:26.215281 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:26.215297 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:26.215897 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:26.215968 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:26.215997 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:26.216093 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:26.216285 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 00:35:26.216418 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":258,"threads_started":1}
I20260812 00:35:26.218158 27514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:26.221650 27519 heartbeater.cc:503] Master 127.7.244.126:42265 was elected leader, sending a full tablet report...
I20260812 00:35:26.299975 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 27278
W20260812 00:35:26.319976 27139 connection.cc:535] server connection from 127.7.244.65:53693 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 00:35:26.320175 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 27527
I20260812 00:35:26.328529 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:26.321107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.7.244.66:39495)
0812 00:35:26.321683 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:26.321690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:26.321719 (+ 29us) server_negotiation.cc:407] Connection header received
0812 00:35:26.321782 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:26.321789 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:26.322005 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:26.322132 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:26.322545 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.323605 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.324966 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.325745 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.325856 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:26.326944 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:26.326954 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:26.326969 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:26.327630 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:26.327683 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:26.327706 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:26.327802 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:26.328028 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 00:35:26.328164 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":303,"threads_started":1}
I20260812 00:35:26.330120 27772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:26.412385 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 27527
I20260812 00:35:26.439474 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 27786
I20260812 00:35:26.447860 28087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:26.440433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.7.244.67:33039)
0812 00:35:26.441041 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:26.441048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:26.441081 (+ 33us) server_negotiation.cc:407] Connection header received
0812 00:35:26.441145 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:26.441152 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:26.441431 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:26.441564 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:26.442020 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.443096 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.444335 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:26.444952 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:26.445066 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:26.446341 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:26.446352 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:26.446369 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:26.446990 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:26.447043 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:26.447066 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:26.447176 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:26.447346 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 00:35:26.447473 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":289,"threads_started":1}
I20260812 00:35:26.449316 28047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:26.532104 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 27786
I20260812 00:35:26.560352 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 27105
I20260812 00:35:26.561691 27221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:26.666103 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 27105
2026-08-12T00:35:26Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2680 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T00:35:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:35:26Z Disabled control of system clock
I20260812 00:35:26.758399 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33123
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:33123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 00:35:27.117987 28101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:27.118340 28101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:27.118450 28101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:27.126103 28101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:35:27.126231 28101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:27.126269 28101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:27.126360 28101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:35:27.126425 28101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:35:27.136817 28101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:33123
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:27.138304 28101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:27.140290 28101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:27.153075 28118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:27.153041 28110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:27.153345 28121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:27.154143 28101 server_base.cc:1034] running on GCE node
I20260812 00:35:27.155143 28101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:27.156978 28101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:27.158252 28101 hybrid_clock.cc:648] HybridClock initialized: now 1786494927158173 us; error 83 us; skew 500 ppm
I20260812 00:35:27.158725 28101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:35:27.161567 28101 webserver.cc:460] Webserver started at http://127.7.244.126:37817/ using document root <none> and password file <none>
I20260812 00:35:27.162379 28101 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:27.162508 28101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:27.162851 28101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:27.166239 28101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "52494fd11ed948c5871af6591c943460"
format_stamp: "Formatted at 2026-08-12 00:35:27 on dist-test-slave-gjw7"
server_key: "4951d8273e39fee94eb8ed0f8d64884e"
server_key_iv: "8b29bd2b020fe89eb5753cf286f68278"
server_key_version: "encryptionkey@0"
I20260812 00:35:27.167202 28101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "52494fd11ed948c5871af6591c943460"
format_stamp: "Formatted at 2026-08-12 00:35:27 on dist-test-slave-gjw7"
server_key: "4951d8273e39fee94eb8ed0f8d64884e"
server_key_iv: "8b29bd2b020fe89eb5753cf286f68278"
server_key_version: "encryptionkey@0"
I20260812 00:35:27.173130 28101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 00:35:27.177352 28131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:27.179025 28101 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 00:35:27.179246 28101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "52494fd11ed948c5871af6591c943460"
format_stamp: "Formatted at 2026-08-12 00:35:27 on dist-test-slave-gjw7"
server_key: "4951d8273e39fee94eb8ed0f8d64884e"
server_key_iv: "8b29bd2b020fe89eb5753cf286f68278"
server_key_version: "encryptionkey@0"
I20260812 00:35:27.179523 28101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:27.186044 28101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:27.186897 28101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:27.187188 28101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:27.206249 28101 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:33123
I20260812 00:35:27.206287 28233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:33123 every 8 connection(s)
I20260812 00:35:27.207749 28101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:35:27.212060 28234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:35:27.217504 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 28101
I20260812 00:35:27.217792 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:35:27.218597 8145 external_mini_cluster.cc:1428] Setting key 637bf20d1413d4c36492c725a74ea264
I20260812 00:35:27.225220 28234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460: Bootstrap starting.
I20260812 00:35:27.228330 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:27.220958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53574 (local address 127.7.244.126:33123)
0812 00:35:27.222167 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:27.222179 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:27.222229 (+ 50us) server_negotiation.cc:407] Connection header received
0812 00:35:27.224109 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:27.224140 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:27.224657 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:27.225107 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:27.226513 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:27.226520 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:27.226535 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:27.226560 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:27.226634 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:27.226653 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:27.226902 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:27.227064 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 00:35:27.227349 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":445,"threads_started":1}
I20260812 00:35:27.229089 28234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460: Neither blocks nor log segments found. Creating new log.
I20260812 00:35:27.230408 28234 log.cc:824] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460: Log is configured to *not* fsync() on all Append() calls
I20260812 00:35:27.233727 28234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460: No bootstrap required, opened a new log
I20260812 00:35:27.241712 28234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52494fd11ed948c5871af6591c943460" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33123 } }
I20260812 00:35:27.242161 28234 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:35:27.242269 28234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52494fd11ed948c5871af6591c943460, State: Initialized, Role: FOLLOWER
I20260812 00:35:27.242872 28234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52494fd11ed948c5871af6591c943460" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33123 } }
I20260812 00:35:27.243122 28234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:35:27.243265 28234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:35:27.243434 28234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:35:27.245870 28234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52494fd11ed948c5871af6591c943460" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33123 } }
I20260812 00:35:27.246464 28234 leader_election.cc:302] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52494fd11ed948c5871af6591c943460; no voters:
I20260812 00:35:27.247128 28234 leader_election.cc:288] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:35:27.247303 28240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:35:27.248504 28240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [term 1 LEADER]: Becoming Leader. State: Replica: 52494fd11ed948c5871af6591c943460, State: Running, Role: LEADER
I20260812 00:35:27.249214 28240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52494fd11ed948c5871af6591c943460" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33123 } }
I20260812 00:35:27.249500 28234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:35:27.252698 28242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52494fd11ed948c5871af6591c943460. Latest consensus state: current_term: 1 leader_uuid: "52494fd11ed948c5871af6591c943460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52494fd11ed948c5871af6591c943460" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33123 } } }
I20260812 00:35:27.253134 28242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:27.253798 28241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52494fd11ed948c5871af6591c943460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52494fd11ed948c5871af6591c943460" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33123 } } }
I20260812 00:35:27.254122 28241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:27.259107 28258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:35:27.264328 28258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:35:27.275280 28258 catalog_manager.cc:1347] Generated new cluster ID: 9644a2bd2ef04f368ea1a7e6b478c000
I20260812 00:35:27.275436 28258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:35:27.296188 28258 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:35:27.296664 28258 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:35:27.304701 28258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52494fd11ed948c5871af6591c943460: Generated new TSK 0
I20260812 00:35:27.305776 28258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:35:27.320360 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33123
--builtin_ntp_servers=127.7.244.84:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 00:35:27.705175 28278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:27.705525 28278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:27.705626 28278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:27.713128 28278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:27.713261 28278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:27.713449 28278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:35:27.723068 28278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:27.724478 28278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:27.726583 28278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:27.741031 28292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:27.740957 28288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:27.741230 28294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:27.741941 28278 server_base.cc:1034] running on GCE node
I20260812 00:35:27.742666 28278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:27.743773 28278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:27.745004 28278 hybrid_clock.cc:648] HybridClock initialized: now 1786494927744948 us; error 44 us; skew 500 ppm
I20260812 00:35:27.745420 28278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:35:27.748214 28278 webserver.cc:460] Webserver started at http://127.7.244.65:42507/ using document root <none> and password file <none>
I20260812 00:35:27.749042 28278 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:27.749150 28278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:27.749604 28278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:27.753078 28278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "4ed946cbf78a4125a78fc2c2b837d530"
format_stamp: "Formatted at 2026-08-12 00:35:27 on dist-test-slave-gjw7"
server_key: "8312a710bfa5c0db17c514f190188799"
server_key_iv: "3ac42f2dbcbcf3b5c32c9e7fef5bbb7c"
server_key_version: "encryptionkey@0"
I20260812 00:35:27.754165 28278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "4ed946cbf78a4125a78fc2c2b837d530"
format_stamp: "Formatted at 2026-08-12 00:35:27 on dist-test-slave-gjw7"
server_key: "8312a710bfa5c0db17c514f190188799"
server_key_iv: "3ac42f2dbcbcf3b5c32c9e7fef5bbb7c"
server_key_version: "encryptionkey@0"
I20260812 00:35:27.760479 28278 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:35:27.764784 28304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:27.766562 28278 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:35:27.766768 28278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "4ed946cbf78a4125a78fc2c2b837d530"
format_stamp: "Formatted at 2026-08-12 00:35:27 on dist-test-slave-gjw7"
server_key: "8312a710bfa5c0db17c514f190188799"
server_key_iv: "3ac42f2dbcbcf3b5c32c9e7fef5bbb7c"
server_key_version: "encryptionkey@0"
I20260812 00:35:27.767027 28278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:27.773411 28278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:27.774326 28278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:27.774595 28278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:27.775658 28278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:27.777283 28278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:27.777431 28278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 00:35:27.777556 28278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:27.777633 28278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:27.810756 28278 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:33583
I20260812 00:35:27.810808 28558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:33583 every 8 connection(s)
I20260812 00:35:27.812191 28278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:35:27.818935 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 28278
I20260812 00:35:27.819186 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:35:27.819962 8145 external_mini_cluster.cc:1428] Setting key a9388d3a958feaf13def3edbba32adb3
I20260812 00:35:27.821621 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:27.814947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:47903 (local address 127.7.244.126:33123)
0812 00:35:27.815425 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:27.815432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:27.816809 (+ 1377us) server_negotiation.cc:407] Connection header received
0812 00:35:27.818703 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:27.818708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:27.818794 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:27.818896 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:27.820667 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:27.820674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:27.820678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:27.820705 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:27.820768 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:27.820787 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:27.820938 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:27.821219 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 00:35:27.821315 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":211,"threads_started":1}
I20260812 00:35:27.822824 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:27.815587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33123 (local address 127.7.244.65:47903)
0812 00:35:27.816619 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:27.816667 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:27.818338 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:27.819257 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:27.819271 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:27.819414 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:27.820221 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:35:27.820225 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:27.820284 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:35:27.820419 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:27.820951 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:27.820966 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:27.820975 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 00:35:27.821329 (+ 354us) client_negotiation.cc:240] Negotiation successful
0812 00:35:27.821595 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":328,"threads_started":1}
I20260812 00:35:27.823550 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33123
--builtin_ntp_servers=127.7.244.84:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 00:35:27.824546 28561 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33123
I20260812 00:35:27.824939 28561 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:27.825764 28561 heartbeater.cc:511] Master 127.7.244.126:33123 requested a full tablet report, sending...
I20260812 00:35:27.828311 28160 ts_manager.cc:194] Registered new tserver with Master: 4ed946cbf78a4125a78fc2c2b837d530 (127.7.244.65:33583)
W20260812 00:35:28.184026 28568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:28.184351 28568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:28.184542 28568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:28.192090 28568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:28.192207 28568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:28.192368 28568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:35:28.202024 28568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:28.203420 28568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:28.205267 28568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:28.219475 28580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:28.219506 28582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:28.222424 28586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:28.223100 28568 server_base.cc:1034] running on GCE node
I20260812 00:35:28.223845 28568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:28.224951 28568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:28.226217 28568 hybrid_clock.cc:648] HybridClock initialized: now 1786494928226106 us; error 94 us; skew 500 ppm
I20260812 00:35:28.226619 28568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:35:28.229336 28568 webserver.cc:460] Webserver started at http://127.7.244.66:37485/ using document root <none> and password file <none>
I20260812 00:35:28.230110 28568 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:28.230239 28568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:28.230605 28568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:28.233894 28568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "d9d4b6a12f75477abdfe6a27cce04ab5"
format_stamp: "Formatted at 2026-08-12 00:35:28 on dist-test-slave-gjw7"
server_key: "b2c8a0864a1c7d3f76855dbfa8e143a9"
server_key_iv: "1378e210591e4ee671e515ccf663643a"
server_key_version: "encryptionkey@0"
I20260812 00:35:28.234822 28568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "d9d4b6a12f75477abdfe6a27cce04ab5"
format_stamp: "Formatted at 2026-08-12 00:35:28 on dist-test-slave-gjw7"
server_key: "b2c8a0864a1c7d3f76855dbfa8e143a9"
server_key_iv: "1378e210591e4ee671e515ccf663643a"
server_key_version: "encryptionkey@0"
I20260812 00:35:28.240854 28568 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:35:28.244984 28596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:28.246542 28568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:35:28.246740 28568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "d9d4b6a12f75477abdfe6a27cce04ab5"
format_stamp: "Formatted at 2026-08-12 00:35:28 on dist-test-slave-gjw7"
server_key: "b2c8a0864a1c7d3f76855dbfa8e143a9"
server_key_iv: "1378e210591e4ee671e515ccf663643a"
server_key_version: "encryptionkey@0"
I20260812 00:35:28.246981 28568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:28.253427 28568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:28.254330 28568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:28.254612 28568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:28.255613 28568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:28.257189 28568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:28.257321 28568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:28.257450 28568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:28.257530 28568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:28.291738 28568 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:46273
I20260812 00:35:28.291790 28824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:46273 every 8 connection(s)
I20260812 00:35:28.293243 28568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:35:28.293588 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 28568
I20260812 00:35:28.293761 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:35:28.294304 8145 external_mini_cluster.cc:1428] Setting key 98e28aac603657155caf779582cb6983
I20260812 00:35:28.297622 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33123
--builtin_ntp_servers=127.7.244.84:41937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 00:35:28.302115 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:28.296178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:51001 (local address 127.7.244.126:33123)
0812 00:35:28.296373 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:28.296380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:28.298110 (+ 1730us) server_negotiation.cc:407] Connection header received
0812 00:35:28.299859 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:28.299865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:28.299945 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:28.300053 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:28.301404 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:28.301410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:28.301414 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:28.301435 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:28.301476 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:28.301491 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:28.301624 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:28.301777 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:35:28.301846 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 00:35:28.303036 28833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:28.296758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33123 (local address 127.7.244.66:51001)
0812 00:35:28.297928 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:28.297976 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:28.299555 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:28.300254 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:28.300263 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:28.300364 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:28.301075 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:35:28.301077 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:28.301120 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:35:28.301213 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:28.301577 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:28.301584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:28.301590 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:35:28.301808 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 00:35:28.302033 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":328,"threads_started":1}
I20260812 00:35:28.304534 28826 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33123
I20260812 00:35:28.304870 28826 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:28.305536 28826 heartbeater.cc:511] Master 127.7.244.126:33123 requested a full tablet report, sending...
I20260812 00:35:28.307026 28160 ts_manager.cc:194] Registered new tserver with Master: d9d4b6a12f75477abdfe6a27cce04ab5 (127.7.244.66:46273)
W20260812 00:35:28.666890 28836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:28.667266 28836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:28.667425 28836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:28.675276 28836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:28.675439 28836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:28.675608 28836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:35:28.686453 28836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:28.687841 28836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:28.689988 28836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:28.705706 28843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:28.705775 28853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:28.705751 28848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:28.706437 28836 server_base.cc:1034] running on GCE node
I20260812 00:35:28.707139 28836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:28.708166 28836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:28.709401 28836 hybrid_clock.cc:648] HybridClock initialized: now 1786494928709343 us; error 45 us; skew 500 ppm
I20260812 00:35:28.709748 28836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:35:28.712292 28836 webserver.cc:460] Webserver started at http://127.7.244.67:42595/ using document root <none> and password file <none>
I20260812 00:35:28.713074 28836 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:28.713182 28836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:28.713567 28836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:28.716770 28836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "5988afa618f5417e9c0cd4df9decc9f5"
format_stamp: "Formatted at 2026-08-12 00:35:28 on dist-test-slave-gjw7"
server_key: "8a81004b3cb48f3ae18ba7666d1015ac"
server_key_iv: "627f2782176362b76eae331135c735e2"
server_key_version: "encryptionkey@0"
I20260812 00:35:28.717751 28836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "5988afa618f5417e9c0cd4df9decc9f5"
format_stamp: "Formatted at 2026-08-12 00:35:28 on dist-test-slave-gjw7"
server_key: "8a81004b3cb48f3ae18ba7666d1015ac"
server_key_iv: "627f2782176362b76eae331135c735e2"
server_key_version: "encryptionkey@0"
I20260812 00:35:28.723887 28836 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 00:35:28.728225 28866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:28.729957 28836 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 00:35:28.730157 28836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "5988afa618f5417e9c0cd4df9decc9f5"
format_stamp: "Formatted at 2026-08-12 00:35:28 on dist-test-slave-gjw7"
server_key: "8a81004b3cb48f3ae18ba7666d1015ac"
server_key_iv: "627f2782176362b76eae331135c735e2"
server_key_version: "encryptionkey@0"
I20260812 00:35:28.730453 28836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:28.736796 28836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:28.737919 28836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:28.738245 28836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:28.739312 28836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:28.740964 28836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:28.741073 28836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:28.741204 28836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:28.741284 28836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:28.773597 28836 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:40301
I20260812 00:35:28.773655 29110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:40301 every 8 connection(s)
I20260812 00:35:28.774986 28836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:35:28.778295 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 28836
I20260812 00:35:28.778509 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:35:28.779093 8145 external_mini_cluster.cc:1428] Setting key a0ab2a61169ea510cba18d4c473a3f86
I20260812 00:35:28.784209 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:28.777840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:50321 (local address 127.7.244.126:33123)
0812 00:35:28.778036 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:28.778041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:28.779923 (+ 1882us) server_negotiation.cc:407] Connection header received
0812 00:35:28.781885 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:28.781891 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:28.781971 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:28.782080 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:28.783551 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:28.783556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:28.783560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:28.783578 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:28.783615 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:28.783628 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:28.783725 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:28.783892 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 00:35:28.783946 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 00:35:28.785332 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:28.778568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33123 (local address 127.7.244.67:50321)
0812 00:35:28.779699 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:28.779760 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:28.781577 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:28.782359 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:28.782368 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:28.782480 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:28.783192 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:35:28.783195 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:28.783241 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:35:28.783351 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:28.783709 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:28.783716 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:28.783721 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 00:35:28.783985 (+ 264us) client_negotiation.cc:240] Negotiation successful
0812 00:35:28.784229 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":358,"threads_started":1}
I20260812 00:35:28.787055 29112 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33123
I20260812 00:35:28.787448 29112 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:28.788229 29112 heartbeater.cc:511] Master 127.7.244.126:33123 requested a full tablet report, sending...
I20260812 00:35:28.790015 28160 ts_manager.cc:194] Registered new tserver with Master: 5988afa618f5417e9c0cd4df9decc9f5 (127.7.244.67:40301)
I20260812 00:35:28.795173 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 00:35:28.801793 28563 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:35:28.800056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.7.244.126:33123)
0812 00:35:28.800220 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:28.800224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:28.800807 (+ 583us) server_negotiation.cc:407] Connection header received
0812 00:35:28.800998 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:28.801002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:28.801058 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:28.801135 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:28.801505 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53576: BlockingRecv error: recv got EOF from 127.0.0.1:53576 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:35:28.802502 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 28278
I20260812 00:35:28.806632 29127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:28.803686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56302 (local address 127.7.244.65:33583)
0812 00:35:28.804222 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:28.804229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:28.804260 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:35:28.804326 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:28.804333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:28.804560 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:28.804697 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:28.805799 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:28.805806 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:28.805820 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:28.805865 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:28.805914 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:28.805930 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:28.806042 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:28.806213 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 00:35:28.806307 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":255,"threads_started":1}
I20260812 00:35:28.807962 28553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:28.889986 28561 heartbeater.cc:503] Master 127.7.244.126:33123 was elected leader, sending a full tablet report...
I20260812 00:35:28.899224 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 28278
I20260812 00:35:28.930543 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 28568
I20260812 00:35:28.934657 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:28.931541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.7.244.66:46273)
0812 00:35:28.932099 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:28.932107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:28.932138 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:35:28.932212 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:28.932219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:28.932449 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:28.932617 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:28.933777 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:28.933785 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:28.933798 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:28.933821 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:28.933869 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:28.933885 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:28.934029 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:28.934188 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 00:35:28.934309 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":257,"threads_started":1}
I20260812 00:35:28.936122 28816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:29.017583 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 28568
I20260812 00:35:29.047703 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 28836
I20260812 00:35:29.051719 29142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:29.048724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41578 (local address 127.7.244.67:40301)
0812 00:35:29.049460 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:29.049467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:29.049501 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:35:29.049569 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:29.049576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:29.049809 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:29.049948 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:29.050997 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:29.051004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:29.051016 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:35:29.051034 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:29.051077 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:35:29.051091 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:29.051225 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:29.051288 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 00:35:29.051374 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":346,"threads_started":1}
I20260812 00:35:29.053103 29105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:29.135438 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 28836
I20260812 00:35:29.160760 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 28101
I20260812 00:35:29.162047 28223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:29.265367 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 28101
2026-08-12T00:35:29Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2598 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:35:29 dist-test-slave-gjw7 krb5kdc[29154](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:35:29 dist-test-slave-gjw7 krb5kdc[29154](info): set up 2 sockets
Aug 12 00:35:29 dist-test-slave-gjw7 krb5kdc[29154](info): commencing operation
krb5kdc: starting...
W20260812 00:35:30.614585 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:35:30 dist-test-slave-gjw7 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:35:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:35:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:35:30.914372 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:46455
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:46455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:31.272032 29173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:31.272377 29173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:31.272493 29173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:31.280071 29173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:35:31.280216 29173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:31.280283 29173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:31.280367 29173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:35:31.280429 29173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:35:31.290907 29173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:46455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:46455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:31.292469 29173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:31.294004 29173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:31.306769 29186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:31.306793 29188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:31.306879 29194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:31.307648 29173 server_base.cc:1034] running on GCE node
I20260812 00:35:31.308524 29173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:31.310127 29173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:31.311400 29173 hybrid_clock.cc:648] HybridClock initialized: now 1786494931311331 us; error 58 us; skew 500 ppm
I20260812 00:35:31.311833 29173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:31 dist-test-slave-gjw7 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494931, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:31.319057 29173 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:35:31.320812 29173 webserver.cc:460] Webserver started at http://127.7.244.126:43211/ using document root <none> and password file <none>
I20260812 00:35:31.321653 29173 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:31.321790 29173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:31.322124 29173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:31.325541 29173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "d159d308863448df9c9b5337d4dd4cf4"
format_stamp: "Formatted at 2026-08-12 00:35:31 on dist-test-slave-gjw7"
server_key: "5b15b7492cdfa26db7a03cbd62099433"
server_key_iv: "3fcbf633c6555aa61fe22648299aa15b"
server_key_version: "encryptionkey@0"
I20260812 00:35:31.326416 29173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "d159d308863448df9c9b5337d4dd4cf4"
format_stamp: "Formatted at 2026-08-12 00:35:31 on dist-test-slave-gjw7"
server_key: "5b15b7492cdfa26db7a03cbd62099433"
server_key_iv: "3fcbf633c6555aa61fe22648299aa15b"
server_key_version: "encryptionkey@0"
I20260812 00:35:31.332355 29173 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260812 00:35:31.336427 29214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:31.338189 29173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:35:31.338415 29173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "d159d308863448df9c9b5337d4dd4cf4"
format_stamp: "Formatted at 2026-08-12 00:35:31 on dist-test-slave-gjw7"
server_key: "5b15b7492cdfa26db7a03cbd62099433"
server_key_iv: "3fcbf633c6555aa61fe22648299aa15b"
server_key_version: "encryptionkey@0"
I20260812 00:35:31.338649 29173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:31.361375 29173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:31.366247 29173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:31.366559 29173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:31.387075 29173 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:46455
I20260812 00:35:31.387101 29335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:46455 every 8 connection(s)
I20260812 00:35:31.388650 29173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:35:31.392864 29337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:35:31.394083 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 29173
I20260812 00:35:31.394371 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:35:31.395067 8145 external_mini_cluster.cc:1428] Setting key 713f9d6306f588479d8a16974823be19
I20260812 00:35:31.408875 29337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4: Bootstrap starting.
Aug 12 00:35:31 dist-test-slave-gjw7 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:35:31.412685 29337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4: Neither blocks nor log segments found. Creating new log.
I20260812 00:35:31.414211 29337 log.cc:824] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4: Log is configured to *not* fsync() on all Append() calls
I20260812 00:35:31.418318 29337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4: No bootstrap required, opened a new log
I20260812 00:35:31.422504 29341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:31.397440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52644 (local address 127.7.244.126:46455)
0812 00:35:31.398391 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:31.398409 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:31.398468 (+ 59us) server_negotiation.cc:407] Connection header received
0812 00:35:31.400266 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:31.400303 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:31.400924 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:31.401473 (+ 549us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:31.403023 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:31.405368 (+ 2345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:31.407144 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:31.408288 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:31.408528 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:31.413240 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:31.413252 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:31.413275 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:31.413368 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:31.417618 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:31.418579 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:31.418588 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:31.418595 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:31.418811 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:31.419273 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:31.419280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:31.419283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:31.419903 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:31.420254 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:31.420756 (+ 502us) server_negotiation.cc:299] Negotiation successful
0812 00:35:31.421273 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":379,"threads_started":1}
I20260812 00:35:31.427142 29337 raft_consensus.cc:348] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d159d308863448df9c9b5337d4dd4cf4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46455 } }
I20260812 00:35:31.427706 29337 raft_consensus.cc:374] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:35:31.427845 29337 raft_consensus.cc:729] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d159d308863448df9c9b5337d4dd4cf4, State: Initialized, Role: FOLLOWER
I20260812 00:35:31.428715 29337 consensus_queue.cc:260] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d159d308863448df9c9b5337d4dd4cf4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46455 } }
I20260812 00:35:31.429024 29337 raft_consensus.cc:388] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:35:31.429169 29337 raft_consensus.cc:482] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:35:31.429370 29337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:35:31.431761 29337 raft_consensus.cc:504] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d159d308863448df9c9b5337d4dd4cf4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46455 } }
I20260812 00:35:31.432361 29337 leader_election.cc:302] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d159d308863448df9c9b5337d4dd4cf4; no voters:
I20260812 00:35:31.433032 29337 leader_election.cc:288] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:35:31.433175 29345 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:35:31.433962 29345 raft_consensus.cc:686] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [term 1 LEADER]: Becoming Leader. State: Replica: d159d308863448df9c9b5337d4dd4cf4, State: Running, Role: LEADER
I20260812 00:35:31.434747 29345 consensus_queue.cc:237] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d159d308863448df9c9b5337d4dd4cf4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46455 } }
I20260812 00:35:31.435492 29337 sys_catalog.cc:564] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:35:31.438000 29352 sys_catalog.cc:455] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d159d308863448df9c9b5337d4dd4cf4. Latest consensus state: current_term: 1 leader_uuid: "d159d308863448df9c9b5337d4dd4cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d159d308863448df9c9b5337d4dd4cf4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46455 } } }
I20260812 00:35:31.438079 29351 sys_catalog.cc:455] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d159d308863448df9c9b5337d4dd4cf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d159d308863448df9c9b5337d4dd4cf4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46455 } } }
I20260812 00:35:31.438529 29352 sys_catalog.cc:458] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:31.438566 29351 sys_catalog.cc:458] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:35:31.446167 29363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:35:31.451113 29363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:35:31.461592 29363 catalog_manager.cc:1347] Generated new cluster ID: d647be4cd1bb44e28d0480f5781cbcb1
I20260812 00:35:31.461736 29363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:35:31.485157 29363 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:35:31.486833 29363 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:35:31.502121 29363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4: Generated new TSK 0
I20260812 00:35:31.503353 29363 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:35:31.575364 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46455
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:31.942939 29391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:31.943261 29391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:31.943408 29391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:31.951105 29391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:31.951244 29391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:31.951396 29391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:35:31.961887 29391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:31.963320 29391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:31.964733 29391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:31.979128 29400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:31.979067 29406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:31.979171 29404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:31.979815 29391 server_base.cc:1034] running on GCE node
I20260812 00:35:31.980602 29391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:31.981745 29391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:31.982986 29391 hybrid_clock.cc:648] HybridClock initialized: now 1786494931982927 us; error 50 us; skew 500 ppm
I20260812 00:35:31.983336 29391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:31 dist-test-slave-gjw7 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494931, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:31.986784 29391 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:35:31.988368 29391 webserver.cc:460] Webserver started at http://127.7.244.65:46351/ using document root <none> and password file <none>
I20260812 00:35:31.989145 29391 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:31.989274 29391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:31.989624 29391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:31.992807 29391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "21ec002372fc4df9b1693ee0cf90fbeb"
format_stamp: "Formatted at 2026-08-12 00:35:31 on dist-test-slave-gjw7"
server_key: "4b80c1fe2785fbe507532f08b2880d10"
server_key_iv: "b1e7605fe750b498a2b4b512775204ed"
server_key_version: "encryptionkey@0"
I20260812 00:35:31.993762 29391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "21ec002372fc4df9b1693ee0cf90fbeb"
format_stamp: "Formatted at 2026-08-12 00:35:31 on dist-test-slave-gjw7"
server_key: "4b80c1fe2785fbe507532f08b2880d10"
server_key_iv: "b1e7605fe750b498a2b4b512775204ed"
server_key_version: "encryptionkey@0"
I20260812 00:35:31.999590 29391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:35:32.003595 29423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:32.005409 29391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:35:32.005578 29391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "21ec002372fc4df9b1693ee0cf90fbeb"
format_stamp: "Formatted at 2026-08-12 00:35:31 on dist-test-slave-gjw7"
server_key: "4b80c1fe2785fbe507532f08b2880d10"
server_key_iv: "b1e7605fe750b498a2b4b512775204ed"
server_key_version: "encryptionkey@0"
I20260812 00:35:32.005791 29391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:32.025883 29391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:32.030625 29391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:32.030907 29391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:32.031999 29391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:32.033564 29391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:32.033649 29391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:32.033754 29391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:32.033799 29391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:32.066120 29391 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:44923
I20260812 00:35:32.066160 29656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:44923 every 8 connection(s)
I20260812 00:35:32.067497 29391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:35:32.076200 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 29391
I20260812 00:35:32.076484 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:35:32.077324 8145 external_mini_cluster.cc:1428] Setting key 61aaebd40dafd1cf2d79052298a2273a
Aug 12 00:35:32 dist-test-slave-gjw7 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494931, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:35:32.090914 29660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:32.070336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:59705 (local address 127.7.244.126:46455)
0812 00:35:32.070855 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:32.070862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:32.071903 (+ 1041us) server_negotiation.cc:407] Connection header received
0812 00:35:32.073917 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:32.073923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:32.074010 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:32.074121 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:32.075974 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:32.077111 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:32.079123 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:32.079676 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:32.079833 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:32.084064 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:32.084071 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:32.084075 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:32.084124 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:32.087420 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:32.088409 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:32.088414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:32.088417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:32.088585 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:32.089056 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:32.089060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:32.089064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:32.089459 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:32.089606 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:32.090391 (+ 785us) server_negotiation.cc:299] Negotiation successful
0812 00:35:32.090640 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":238,"threads_started":1}
I20260812 00:35:32.092315 29662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:32.070900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46455 (local address 127.7.244.65:59705)
0812 00:35:32.071703 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:32.071759 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:32.073561 (+ 1802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:32.074468 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:32.074478 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:32.075053 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:32.075735 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:32.075758 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:32.077388 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:32.077394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:32.078956 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:32.078974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:32.079954 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:32.079960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:32.080193 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:32.080991 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:32.081022 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:32.083738 (+ 2716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:32.087648 (+ 3910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:32.087657 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:32.087675 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:32.088217 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:32.088706 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:32.088710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:32.088712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:32.088937 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:32.089596 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:32.089603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:32.089984 (+ 381us) client_negotiation.cc:769] Sending connection context
0812 00:35:32.090264 (+ 280us) client_negotiation.cc:240] Negotiation successful
0812 00:35:32.090781 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":311,"threads_started":1}
I20260812 00:35:32.094185 29657 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46455
I20260812 00:35:32.094547 29657 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:32.095507 29657 heartbeater.cc:511] Master 127.7.244.126:46455 requested a full tablet report, sending...
I20260812 00:35:32.098183 29254 ts_manager.cc:194] Registered new tserver with Master: 21ec002372fc4df9b1693ee0cf90fbeb (127.7.244.65:44923)
I20260812 00:35:32.099749 29254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:59705
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:35:32.140965 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46455
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:32.503613 29666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:32.503948 29666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:32.504081 29666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:32.511757 29666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:32.511924 29666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:32.512085 29666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:35:32.522555 29666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:32.524003 29666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:32.525538 29666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:35:32.540019 29672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:32.540505 29673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:32.540550 29678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:32.541567 29666 server_base.cc:1034] running on GCE node
I20260812 00:35:32.542268 29666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:32.543365 29666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:32.544579 29666 hybrid_clock.cc:648] HybridClock initialized: now 1786494932544484 us; error 111 us; skew 500 ppm
I20260812 00:35:32.544925 29666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:32 dist-test-slave-gjw7 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:32.548493 29666 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:35:32.550235 29666 webserver.cc:460] Webserver started at http://127.7.244.66:45093/ using document root <none> and password file <none>
I20260812 00:35:32.551017 29666 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:32.551152 29666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:32.551501 29666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:32.554909 29666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "ccb364fa627e4117bc404e2af2215ecf"
format_stamp: "Formatted at 2026-08-12 00:35:32 on dist-test-slave-gjw7"
server_key: "6e45253c0baa924c2cc3d0c580867705"
server_key_iv: "797d247340a7b4d5e3bcdfcfd89c5ab6"
server_key_version: "encryptionkey@0"
I20260812 00:35:32.555936 29666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "ccb364fa627e4117bc404e2af2215ecf"
format_stamp: "Formatted at 2026-08-12 00:35:32 on dist-test-slave-gjw7"
server_key: "6e45253c0baa924c2cc3d0c580867705"
server_key_iv: "797d247340a7b4d5e3bcdfcfd89c5ab6"
server_key_version: "encryptionkey@0"
I20260812 00:35:32.561962 29666 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260812 00:35:32.565900 29694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:32.567456 29666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:35:32.567644 29666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "ccb364fa627e4117bc404e2af2215ecf"
format_stamp: "Formatted at 2026-08-12 00:35:32 on dist-test-slave-gjw7"
server_key: "6e45253c0baa924c2cc3d0c580867705"
server_key_iv: "797d247340a7b4d5e3bcdfcfd89c5ab6"
server_key_version: "encryptionkey@0"
I20260812 00:35:32.567858 29666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:32.587711 29666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:32.592561 29666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:32.592940 29666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:32.594218 29666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:32.595726 29666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:32.595805 29666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:32.595934 29666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:32.595981 29666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:32.628625 29666 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:46191
I20260812 00:35:32.628665 29923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:46191 every 8 connection(s)
I20260812 00:35:32.630067 29666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:35:32.630980 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 29666
I20260812 00:35:32.631182 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:35:32.631760 8145 external_mini_cluster.cc:1428] Setting key 446f0f162180b86606e9faefaaac5d2f
Aug 12 00:35:32 dist-test-slave-gjw7 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494932, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:35:32.653590 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:32.633057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:47563 (local address 127.7.244.126:46455)
0812 00:35:32.633535 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:32.633539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:32.634897 (+ 1358us) server_negotiation.cc:407] Connection header received
0812 00:35:32.636914 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:32.636920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:32.637011 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:32.637129 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:32.638829 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:32.639834 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:32.641725 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:32.642318 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:32.642456 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:32.646763 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:32.646772 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:32.646777 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:32.646822 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:32.649637 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:32.650756 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:32.650760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:32.650762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:32.650919 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:32.651501 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:32.651504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:32.651507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:32.651828 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:32.651958 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:32.653020 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0812 00:35:32.653250 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":264,"threads_started":1}
I20260812 00:35:32.655102 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:32.633865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46455 (local address 127.7.244.66:47563)
0812 00:35:32.634717 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:32.634767 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:32.636596 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:32.637393 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:32.637402 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:32.638002 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:32.638617 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:32.638636 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:32.640052 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:32.640057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:32.641518 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:32.641536 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:32.642600 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:32.642608 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:32.642841 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:32.643574 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:32.643610 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:32.646368 (+ 2758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:32.649861 (+ 3493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:32.649872 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:32.649893 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:32.650585 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:32.651057 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:32.651062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:32.651066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:32.651375 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:32.651971 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:32.651981 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:32.652531 (+ 550us) client_negotiation.cc:769] Sending connection context
0812 00:35:32.652898 (+ 367us) client_negotiation.cc:240] Negotiation successful
0812 00:35:32.653567 (+ 669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":333,"threads_started":1}
I20260812 00:35:32.656942 29929 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46455
I20260812 00:35:32.657375 29929 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:32.658479 29929 heartbeater.cc:511] Master 127.7.244.126:46455 requested a full tablet report, sending...
I20260812 00:35:32.660835 29254 ts_manager.cc:194] Registered new tserver with Master: ccb364fa627e4117bc404e2af2215ecf (127.7.244.66:46191)
I20260812 00:35:32.662078 29254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:47563
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:35:32.705405 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46455
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:35:33.071856 29948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:35:33.072197 29948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:35:33.072382 29948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:35:33.080207 29948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:35:33.080389 29948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:35:33.080547 29948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:35:33.091384 29948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:35:33.092865 29948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:35:33.094409 29948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:35:33.103941 29657 heartbeater.cc:503] Master 127.7.244.126:46455 was elected leader, sending a full tablet report...
W20260812 00:35:33.109364 29963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:33.109346 29965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:35:33.109393 29967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:35:33.110824 29948 server_base.cc:1034] running on GCE node
I20260812 00:35:33.111590 29948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:35:33.112669 29948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:35:33.113899 29948 hybrid_clock.cc:648] HybridClock initialized: now 1786494933113838 us; error 49 us; skew 500 ppm
I20260812 00:35:33.114281 29948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:35:33 dist-test-slave-gjw7 krb5kdc[29154](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494933, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:35:33.117791 29948 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:35:33.119565 29948 webserver.cc:460] Webserver started at http://127.7.244.67:32913/ using document root <none> and password file <none>
I20260812 00:35:33.120333 29948 fs_manager.cc:359] Metadata directory not provided
I20260812 00:35:33.120452 29948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:35:33.120752 29948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:35:33.124001 29948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "ff52aca3236045af9e6ae102f1af681c"
format_stamp: "Formatted at 2026-08-12 00:35:33 on dist-test-slave-gjw7"
server_key: "3e7ec679fc47a61d72a5143b98ebab31"
server_key_iv: "6689396dd03ca46885b86536f24903c2"
server_key_version: "encryptionkey@0"
I20260812 00:35:33.124836 29948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "ff52aca3236045af9e6ae102f1af681c"
format_stamp: "Formatted at 2026-08-12 00:35:33 on dist-test-slave-gjw7"
server_key: "3e7ec679fc47a61d72a5143b98ebab31"
server_key_iv: "6689396dd03ca46885b86536f24903c2"
server_key_version: "encryptionkey@0"
I20260812 00:35:33.131079 29948 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 00:35:33.135267 29981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:33.136831 29948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 00:35:33.137027 29948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "ff52aca3236045af9e6ae102f1af681c"
format_stamp: "Formatted at 2026-08-12 00:35:33 on dist-test-slave-gjw7"
server_key: "3e7ec679fc47a61d72a5143b98ebab31"
server_key_iv: "6689396dd03ca46885b86536f24903c2"
server_key_version: "encryptionkey@0"
I20260812 00:35:33.137244 29948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:35:33.151654 29948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:35:33.157869 29948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:35:33.158174 29948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:35:33.159267 29948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:35:33.160800 29948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:35:33.160888 29948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:33.160981 29948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:35:33.161026 29948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:35:33.193807 29948 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:45753
I20260812 00:35:33.193862 30238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:45753 every 8 connection(s)
I20260812 00:35:33.195264 29948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:35:33.195569 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 29948
I20260812 00:35:33.195791 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:35:33.196445 8145 external_mini_cluster.cc:1428] Setting key 1454ec53d66d8c37588f3e11b2c1811b
Aug 12 00:35:33 dist-test-slave-gjw7 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494933, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:35:33.218750 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.198499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:53559 (local address 127.7.244.126:46455)
0812 00:35:33.201665 (+ 3166us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.201670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.201689 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:35:33.202709 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.202713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.202790 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.202889 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:33.204638 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.205508 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.207272 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.207751 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.207885 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.211999 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:33.212004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:33.212007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:33.212061 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:33.215182 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.216154 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.216159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.216161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.216323 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.216876 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.216879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.216881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.217202 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:33.217373 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.218219 (+ 846us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.218436 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3054,"thread_start_us":2955,"threads_started":1}
I20260812 00:35:33.219813 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.199124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46455 (local address 127.7.244.67:53559)
0812 00:35:33.200178 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:33.200247 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:33.202416 (+ 2169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:33.203115 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:33.203124 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:33.203732 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:35:33.204450 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.204472 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.205678 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.205682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.207028 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.207052 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.208023 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.208030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.208269 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.209119 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:35:33.209155 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:35:33.211772 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:35:33.215410 (+ 3638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:33.215422 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:33.215443 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:33.216016 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:33.216502 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:35:33.216507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:35:33.216510 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:35:33.216771 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:35:33.217417 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:35:33.217425 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:35:33.217841 (+ 416us) client_negotiation.cc:769] Sending connection context
0812 00:35:33.218118 (+ 277us) client_negotiation.cc:240] Negotiation successful
0812 00:35:33.218605 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":408,"threads_started":1}
I20260812 00:35:33.221457 30239 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46455
I20260812 00:35:33.221812 30239 heartbeater.cc:464] Registering TS with master...
I20260812 00:35:33.222643 30239 heartbeater.cc:511] Master 127.7.244.126:46455 requested a full tablet report, sending...
I20260812 00:35:33.224521 29254 ts_manager.cc:194] Registered new tserver with Master: ff52aca3236045af9e6ae102f1af681c (127.7.244.67:45753)
I20260812 00:35:33.225749 29254 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:53559
I20260812 00:35:33.226413 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:35:33.245780 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.231423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.7.244.126:46455)
0812 00:35:33.231688 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.231695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.232065 (+ 370us) server_negotiation.cc:407] Connection header received
0812 00:35:33.232243 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.232247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.232324 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.232454 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:33.234204 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.235016 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.236568 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.237006 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.237118 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.239780 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:33.239786 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:33.239790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:33.239838 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:33.242845 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.243730 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.243734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.243737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.243898 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.244239 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.244242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.244244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.244637 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:33.244832 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.245172 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.245448 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260812 00:35:33.250643 29254 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:35:33.257872 29254 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:35:33.297441 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.273349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.7.244.67:45753)
0812 00:35:33.287781 (+ 14432us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.287805 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.288781 (+ 976us) server_negotiation.cc:407] Connection header received
0812 00:35:33.288902 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.288911 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.289225 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.289467 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:35:33.290538 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.291599 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.294479 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.296001 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.296159 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.296446 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.296913 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.297105 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14204,"thread_start_us":14472,"threads_started":1}
I20260812 00:35:33.297546 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.273014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:45753 (local address 127.0.0.1:46690)
0812 00:35:33.288475 (+ 15461us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:33.288498 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:33.288647 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:33.289760 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:33.289765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:33.289788 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:35:33.290357 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.290373 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.291751 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.291758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.294277 (+ 2519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.294292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.296250 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.296257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.296485 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.296568 (+ 83us) client_negotiation.cc:769] Sending connection context
0812 00:35:33.296718 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 00:35:33.296964 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15330,"mutex_wait_us":203,"thread_start_us":15195,"threads_started":1}
I20260812 00:35:33.297621 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.272089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:46191 (local address 127.0.0.1:52042)
0812 00:35:33.280447 (+ 8358us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:33.280473 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:33.280632 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:33.289485 (+ 8853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:33.289493 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:33.289528 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:35:33.290106 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.290119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.292060 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.292066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.294667 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.294684 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.296653 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.296660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.296866 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.296897 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 00:35:33.296981 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:35:33.297128 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8191,"mutex_wait_us":59,"thread_start_us":8004,"threads_started":1}
I20260812 00:35:33.298652 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.272593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52042 (local address 127.7.244.66:46191)
0812 00:35:33.288696 (+ 16103us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.288707 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.288749 (+ 42us) server_negotiation.cc:407] Connection header received
0812 00:35:33.288891 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.288902 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.289213 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.289465 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:35:33.290371 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.291896 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.294868 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.296353 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.296637 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.298089 (+ 1452us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.298170 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.298381 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15883,"thread_start_us":15839,"threads_started":1}
I20260812 00:35:33.299093 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.271795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.7.244.65:44923)
0812 00:35:33.289269 (+ 17474us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.289277 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.289337 (+ 60us) server_negotiation.cc:407] Connection header received
0812 00:35:33.289442 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.289449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.289683 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.289892 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:35:33.290751 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.292780 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.295266 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.296403 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.296543 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.296775 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.298620 (+ 1845us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.298816 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17290,"thread_start_us":17130,"threads_started":1}
I20260812 00:35:33.301826 30088 tablet_service.cc:1467] Processing CreateTablet for tablet dc8d7ae3a1724ffc954d1937d0a5c47b (DEFAULT_TABLE table=test-table [id=9cab5b25322d44c38d62836c4a407e4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:35:33.302197 29797 tablet_service.cc:1467] Processing CreateTablet for tablet dc8d7ae3a1724ffc954d1937d0a5c47b (DEFAULT_TABLE table=test-table [id=9cab5b25322d44c38d62836c4a407e4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:35:33.303020 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.271638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44923 (local address 127.0.0.1:55892)
0812 00:35:33.279124 (+ 7486us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:33.279208 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:33.279486 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:33.289916 (+ 10430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:33.289921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:33.289958 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:35:33.290530 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.290544 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.292966 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.292972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.295089 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.295100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.298209 (+ 3109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.298220 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.298400 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.298435 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:35:33.298536 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 00:35:33.298671 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":29,"thread_start_us":432,"threads_started":1}
I20260812 00:35:33.303498 30088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc8d7ae3a1724ffc954d1937d0a5c47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:35:33.303565 29797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc8d7ae3a1724ffc954d1937d0a5c47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:35:33.307372 29529 tablet_service.cc:1467] Processing CreateTablet for tablet dc8d7ae3a1724ffc954d1937d0a5c47b (DEFAULT_TABLE table=test-table [id=9cab5b25322d44c38d62836c4a407e4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:35:33.309015 29529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc8d7ae3a1724ffc954d1937d0a5c47b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:35:33.316306 30299 tablet_bootstrap.cc:492] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Bootstrap starting.
I20260812 00:35:33.318857 30304 tablet_bootstrap.cc:492] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Bootstrap starting.
I20260812 00:35:33.321045 30299 tablet_bootstrap.cc:654] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Neither blocks nor log segments found. Creating new log.
I20260812 00:35:33.322711 30304 tablet_bootstrap.cc:654] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Neither blocks nor log segments found. Creating new log.
I20260812 00:35:33.322710 30299 log.cc:824] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Log is configured to *not* fsync() on all Append() calls
I20260812 00:35:33.323828 30305 tablet_bootstrap.cc:492] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: Bootstrap starting.
I20260812 00:35:33.324223 30304 log.cc:824] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Log is configured to *not* fsync() on all Append() calls
I20260812 00:35:33.327557 30304 tablet_bootstrap.cc:492] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: No bootstrap required, opened a new log
I20260812 00:35:33.327864 30304 ts_tablet_manager.cc:1397] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 00:35:33.327962 30299 tablet_bootstrap.cc:492] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: No bootstrap required, opened a new log
I20260812 00:35:33.328130 30305 tablet_bootstrap.cc:654] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: Neither blocks nor log segments found. Creating new log.
I20260812 00:35:33.328284 30299 ts_tablet_manager.cc:1397] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260812 00:35:33.329622 30305 log.cc:824] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: Log is configured to *not* fsync() on all Append() calls
I20260812 00:35:33.333357 30305 tablet_bootstrap.cc:492] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: No bootstrap required, opened a new log
I20260812 00:35:33.333683 30305 ts_tablet_manager.cc:1397] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260812 00:35:33.336789 30304 raft_consensus.cc:348] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.337248 30304 raft_consensus.cc:374] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:35:33.337388 30304 raft_consensus.cc:729] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb364fa627e4117bc404e2af2215ecf, State: Initialized, Role: FOLLOWER
I20260812 00:35:33.338011 30304 consensus_queue.cc:260] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.338577 30299 raft_consensus.cc:348] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.339028 30299 raft_consensus.cc:374] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:35:33.339116 30299 raft_consensus.cc:729] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff52aca3236045af9e6ae102f1af681c, State: Initialized, Role: FOLLOWER
I20260812 00:35:33.339128 29929 heartbeater.cc:503] Master 127.7.244.126:46455 was elected leader, sending a full tablet report...
I20260812 00:35:33.339717 30299 consensus_queue.cc:260] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.339902 30304 ts_tablet_manager.cc:1428] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260812 00:35:33.341254 30239 heartbeater.cc:503] Master 127.7.244.126:46455 was elected leader, sending a full tablet report...
I20260812 00:35:33.341873 30299 ts_tablet_manager.cc:1428] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260812 00:35:33.344112 30305 raft_consensus.cc:348] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.344553 30305 raft_consensus.cc:374] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:35:33.344658 30305 raft_consensus.cc:729] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21ec002372fc4df9b1693ee0cf90fbeb, State: Initialized, Role: FOLLOWER
I20260812 00:35:33.345340 30305 consensus_queue.cc:260] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.348356 30305 ts_tablet_manager.cc:1428] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
W20260812 00:35:33.382279 29931 tablet.cc:2369] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:35:33.447358 30240 tablet.cc:2369] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:35:33.482512 30325 raft_consensus.cc:482] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:35:33.482852 30325 raft_consensus.cc:504] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.484710 30325 leader_election.cc:288] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff52aca3236045af9e6ae102f1af681c (127.7.244.67:45753), ccb364fa627e4117bc404e2af2215ecf (127.7.244.66:46191)
I20260812 00:35:33.491840 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.485193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:50397 (local address 127.7.244.67:45753)
0812 00:35:33.485459 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.485465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.486290 (+ 825us) server_negotiation.cc:407] Connection header received
0812 00:35:33.486483 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.486487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.486558 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.486650 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:35:33.487292 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.488036 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.490061 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.490838 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.490983 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.491065 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.491454 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.491591 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 00:35:33.491868 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.485336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:48031 (local address 127.7.244.66:46191)
0812 00:35:33.485556 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.485563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.486221 (+ 658us) server_negotiation.cc:407] Connection header received
0812 00:35:33.486402 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.486407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.486482 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.486552 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:35:33.487275 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.487983 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.490069 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.490878 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.491076 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.491152 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.491479 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.491611 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:35:33.491966 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.485192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:45753 (local address 127.7.244.65:50397)
0812 00:35:33.486144 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:33.486161 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:33.486326 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:33.486684 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:33.486689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:33.486719 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:35:33.487110 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.487121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.488170 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.488174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.489897 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.489910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.491081 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.491089 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.491279 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.491309 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 00:35:33.491386 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 00:35:33.491519 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":92,"thread_start_us":577,"threads_started":1}
I20260812 00:35:33.492220 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.485165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:46191 (local address 127.7.244.65:48031)
0812 00:35:33.486081 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0812 00:35:33.486098 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:35:33.486247 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:35:33.486605 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:35:33.486609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:35:33.486636 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:35:33.487114 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.487124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.488176 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.488190 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.489883 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:35:33.489899 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.491125 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.491130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:35:33.491310 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.491336 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 00:35:33.491428 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 00:35:33.491560 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":52,"spinlock_wait_cycles":29440,"thread_start_us":507,"threads_started":1}
I20260812 00:35:33.493033 30141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc8d7ae3a1724ffc954d1937d0a5c47b" candidate_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff52aca3236045af9e6ae102f1af681c" is_pre_election: true
I20260812 00:35:33.493384 29832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc8d7ae3a1724ffc954d1937d0a5c47b" candidate_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb364fa627e4117bc404e2af2215ecf" is_pre_election: true
I20260812 00:35:33.493716 30141 raft_consensus.cc:2454] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21ec002372fc4df9b1693ee0cf90fbeb in term 0.
I20260812 00:35:33.493969 29832 raft_consensus.cc:2454] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21ec002372fc4df9b1693ee0cf90fbeb in term 0.
I20260812 00:35:33.494586 29428 leader_election.cc:302] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 21ec002372fc4df9b1693ee0cf90fbeb, ccb364fa627e4117bc404e2af2215ecf, ff52aca3236045af9e6ae102f1af681c; no voters:
I20260812 00:35:33.495003 30325 raft_consensus.cc:2781] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:35:33.495138 30325 raft_consensus.cc:482] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:35:33.495296 30325 raft_consensus.cc:3037] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:35:33.497697 30325 raft_consensus.cc:504] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.498364 30325 leader_election.cc:288] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [CANDIDATE]: Term 1 election: Requested vote from peers ff52aca3236045af9e6ae102f1af681c (127.7.244.67:45753), ccb364fa627e4117bc404e2af2215ecf (127.7.244.66:46191)
I20260812 00:35:33.498946 29832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc8d7ae3a1724ffc954d1937d0a5c47b" candidate_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb364fa627e4117bc404e2af2215ecf"
I20260812 00:35:33.498948 30141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc8d7ae3a1724ffc954d1937d0a5c47b" candidate_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff52aca3236045af9e6ae102f1af681c"
I20260812 00:35:33.499142 29832 raft_consensus.cc:3037] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:35:33.499161 30141 raft_consensus.cc:3037] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:35:33.501530 29832 raft_consensus.cc:2454] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21ec002372fc4df9b1693ee0cf90fbeb in term 1.
I20260812 00:35:33.501534 30141 raft_consensus.cc:2454] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21ec002372fc4df9b1693ee0cf90fbeb in term 1.
I20260812 00:35:33.502102 29426 leader_election.cc:302] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21ec002372fc4df9b1693ee0cf90fbeb, ccb364fa627e4117bc404e2af2215ecf; no voters:
I20260812 00:35:33.502462 30325 raft_consensus.cc:2781] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:35:33.502972 30325 raft_consensus.cc:686] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 1 LEADER]: Becoming Leader. State: Replica: 21ec002372fc4df9b1693ee0cf90fbeb, State: Running, Role: LEADER
I20260812 00:35:33.503533 30325 consensus_queue.cc:237] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } }
I20260812 00:35:33.509027 29252 catalog_manager.cc:5495] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb reported cstate change: term changed from 0 to 1, leader changed from <none> to 21ec002372fc4df9b1693ee0cf90fbeb (127.7.244.65). New cstate: current_term: 1 leader_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "21ec002372fc4df9b1693ee0cf90fbeb" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260812 00:35:33.535006 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.529490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55900 (local address 127.7.244.65:44923)
0812 00:35:33.529704 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.529710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.529731 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:35:33.529988 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.529992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.530076 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.530230 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:35:33.530633 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.531350 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.532726 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.533141 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.533269 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.533846 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:35:33.534383 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:35:33.534530 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.534639 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.534735 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:35:33.544958 30141 raft_consensus.cc:1261] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 1 FOLLOWER]: Refusing update from remote peer 21ec002372fc4df9b1693ee0cf90fbeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:35:33.544958 29832 raft_consensus.cc:1261] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 1 FOLLOWER]: Refusing update from remote peer 21ec002372fc4df9b1693ee0cf90fbeb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:35:33.546353 30330 consensus_queue.cc:1035] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb364fa627e4117bc404e2af2215ecf" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:35:33.546834 30325 consensus_queue.cc:1035] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff52aca3236045af9e6ae102f1af681c" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:35:33.579450 29252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:35:33.579751 29252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:35:33.583794 29252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d159d308863448df9c9b5337d4dd4cf4: Sending DeleteTablet for 3 replicas of tablet dc8d7ae3a1724ffc954d1937d0a5c47b
I20260812 00:35:33.585325 29797 tablet_service.cc:1514] Processing DeleteTablet for tablet dc8d7ae3a1724ffc954d1937d0a5c47b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:35:33 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:52042
I20260812 00:35:33.585703 29529 tablet_service.cc:1514] Processing DeleteTablet for tablet dc8d7ae3a1724ffc954d1937d0a5c47b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:35:33 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55892
I20260812 00:35:33.586193 30088 tablet_service.cc:1514] Processing DeleteTablet for tablet dc8d7ae3a1724ffc954d1937d0a5c47b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:35:33 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:46690
I20260812 00:35:33.587729 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 29391
I20260812 00:35:33.597167 30354 tablet_replica.cc:330] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: stopping tablet replica
I20260812 00:35:33.597970 30354 raft_consensus.cc:2229] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:35:33.598925 30354 raft_consensus.cc:2258] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:35:33.599901 30353 tablet_replica.cc:330] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: stopping tablet replica
I20260812 00:35:33.600471 30353 raft_consensus.cc:2229] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:35:33.601104 30353 raft_consensus.cc:2258] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:35:33.601768 30354 ts_tablet_manager.cc:1905] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:35:33.604086 30353 ts_tablet_manager.cc:1905] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:35:33.609891 30354 ts_tablet_manager.cc:1918] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:35:33.610119 30354 log.cc:1196] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/dc8d7ae3a1724ffc954d1937d0a5c47b
I20260812 00:35:33.610749 30354 ts_tablet_manager.cc:1939] T dc8d7ae3a1724ffc954d1937d0a5c47b P 21ec002372fc4df9b1693ee0cf90fbeb: Deleting consensus metadata
I20260812 00:35:33.610760 30355 tablet_replica.cc:330] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: stopping tablet replica
I20260812 00:35:33.611454 30355 raft_consensus.cc:2229] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:35:33.612120 30355 raft_consensus.cc:2258] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:35:33.614149 29221 catalog_manager.cc:4841] TS 21ec002372fc4df9b1693ee0cf90fbeb (127.7.244.65:44923): tablet dc8d7ae3a1724ffc954d1937d0a5c47b (table test-table [id=9cab5b25322d44c38d62836c4a407e4d]) successfully deleted
I20260812 00:35:33.615057 30353 ts_tablet_manager.cc:1918] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:35:33.615257 30353 log.cc:1196] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/dc8d7ae3a1724ffc954d1937d0a5c47b
I20260812 00:35:33.615738 30353 ts_tablet_manager.cc:1939] T dc8d7ae3a1724ffc954d1937d0a5c47b P ccb364fa627e4117bc404e2af2215ecf: Deleting consensus metadata
I20260812 00:35:33.616074 30355 ts_tablet_manager.cc:1905] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:35:33.617978 29219 catalog_manager.cc:4841] TS ccb364fa627e4117bc404e2af2215ecf (127.7.244.66:46191): tablet dc8d7ae3a1724ffc954d1937d0a5c47b (table test-table [id=9cab5b25322d44c38d62836c4a407e4d]) successfully deleted
Aug 12 00:35:33 dist-test-slave-gjw7 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:35:33.624920 30355 ts_tablet_manager.cc:1918] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:35:33.625135 30355 log.cc:1196] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/dc8d7ae3a1724ffc954d1937d0a5c47b
I20260812 00:35:33.625825 30355 ts_tablet_manager.cc:1939] T dc8d7ae3a1724ffc954d1937d0a5c47b P ff52aca3236045af9e6ae102f1af681c: Deleting consensus metadata
I20260812 00:35:33.629071 29221 catalog_manager.cc:4841] TS ff52aca3236045af9e6ae102f1af681c (127.7.244.67:45753): tablet dc8d7ae3a1724ffc954d1937d0a5c47b (table test-table [id=9cab5b25322d44c38d62836c4a407e4d]) successfully deleted
I20260812 00:35:33.631919 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.589999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55902 (local address 127.7.244.65:44923)
0812 00:35:33.590205 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.590211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.609759 (+ 19548us) server_negotiation.cc:407] Connection header received
0812 00:35:33.610028 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.610033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.610168 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.610299 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:33.612506 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.613730 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.618983 (+ 5253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.619468 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.619605 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.624050 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:33.624066 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:33.624083 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:33.624129 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:33.627744 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.628969 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.628980 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.628983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.629195 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.629902 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.629908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.629911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.630482 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:33.630819 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.631236 (+ 417us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.631528 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:35:33.632705 29645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:33.737262 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 29391
I20260812 00:35:33.763674 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 29666
Aug 12 00:35:33 dist-test-slave-gjw7 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:35:33.779296 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.764501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.7.244.66:46191)
0812 00:35:33.764678 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.764682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.764702 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:35:33.764845 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.764849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.764931 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.765022 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:33.766324 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.767116 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.768336 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.768756 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.768858 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.773280 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:33.773330 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:33.773350 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:33.773400 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:33.776359 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.777227 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.777234 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.777237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.777473 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.777866 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.777869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.777871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.778268 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:33.778512 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.778771 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.778955 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 00:35:33.780221 29913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:33.872013 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 29666
I20260812 00:35:33.900138 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 29948
Aug 12 00:35:33 dist-test-slave-gjw7 krb5kdc[29154](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:35:33.916114 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:35:33.901013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.7.244.67:45753)
0812 00:35:33.901199 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 00:35:33.901204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:35:33.901225 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:35:33.901409 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:35:33.901413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:35:33.901495 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:35:33.901582 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:35:33.903056 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.903852 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.905256 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:35:33.905746 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:35:33.905847 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:35:33.910589 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:35:33.910610 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:35:33.910630 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:35:33.910678 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:35:33.913509 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.914258 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.914266 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.914269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.914450 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:35:33.914764 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:35:33.914766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:35:33.914768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:35:33.915122 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:35:33.915390 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 00:35:33.915631 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 00:35:33.915805 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 00:35:33.916764 30233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:34.004994 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 29948
I20260812 00:35:34.033737 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 29173
I20260812 00:35:34.034999 29326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:35:34.146687 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 29173
2026-08-12T00:35:34Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4879 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:35:34 dist-test-slave-gjw7 krb5kdc[30369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:35:34 dist-test-slave-gjw7 krb5kdc[30369](info): set up 2 sockets
Aug 12 00:35:34 dist-test-slave-gjw7 krb5kdc[30369](info): commencing operation
krb5kdc: starting...
W20260812 00:35:35.471601 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:35:35 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:35:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:35:35Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.244.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.244.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/rangeradmin_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/rangeradmin_127.7.244.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.244.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.244.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/rangerlookup_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/rangerlookup_127.7.244.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.244.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.244.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/HTTP_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/HTTP_127.7.244.84@KRBTEST.COM.keytab.
I20260812 00:35:35.884696 8145 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOGbh_J/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/postgres -l logfile start
2026-08-12 00:35:38.456 UTC [30435] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 00:35:38.456 UTC [30435] LOG: listening on IPv4 address "127.7.244.84", port 60841
2026-08-12 00:35:38.459 UTC [30435] LOG: listening on Unix socket "/tmp/.s.PGSQL.60841"
2026-08-12 00:35:38.483 UTC [30440] LOG: database system was shut down at 2026-08-12 00:35:36 UTC
2026-08-12 00:35:38.487 UTC [30435] LOG: database system is ready to accept connections
I20260812 00:35:39.661867 8145 mini_postgres.cc:96] Postgres bound to 60841
2026-08-12 00:35:39.684 UTC [30463] FATAL: database "slave" does not exist
127.7.244.84:60841 - accepting connections
I20260812 00:35:39.685829 8145 mini_ranger.cc:161] Starting Ranger...
I20260812 00:35:39.719587 8145 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 00:35:40.227792 8145 mini_ranger.cc:87] Created ranger Postgres database
I20260812 00:35:40.227943 8145 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-admin
2026-08-12 00:35:40,538 [I] DB FLAVOR :POSTGRES
2026-08-12 00:35:40,539 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 00:35:40,539 [I] Checking connection..
2026-08-12 00:35:40,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 00:35:40,821 [I] Checking connection passed.
2026-08-12 00:35:40,821 [I] --------- Verifying version history table ---------
2026-08-12 00:35:40,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:35:41,108 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 00:35:41,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:35:41,363 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 00:35:41,363 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 00:35:41,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 00:35:41.619 UTC [30602] WARNING: there is no transaction in progress
2026-08-12 00:35:41,626 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 00:35:41,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:35:41,897 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 00:35:41,898 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 00:35:41,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 00:35:42,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 00:35:42,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjw7.c.gcp-upstream.internal','N') ;"
2026-08-12 00:35:42,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 00:35:43,005 [I] Table x_portal_user does not exist in database ranger
2026-08-12 00:35:43,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 00:35:43,287 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 00:35:43,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 00:35:43,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 00:35:43,824 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 00:35:43,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 00:35:44.884 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:44.903 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:44.921 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:44.939 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:44.957 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:44.979 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.045 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.053 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.068 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.080 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.091 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.106 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.117 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.125 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.133 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45.141 UTC [30818] WARNING: there is no transaction in progress
2026-08-12 00:35:45,881 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 00:35:45,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 00:35:46,160 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 00:35:46,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 00:35:46,424 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 00:35:46,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 00:35:46,684 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 00:35:46,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 00:35:46,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 00:35:47,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjw7.c.gcp-upstream.internal';"
2026-08-12 00:35:47,490 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 00:35:47,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 00:35:47,779 [I] DB_PATCHES have already been applied
I20260812 00:35:47.785965 8145 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOGbh_J/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 00:35:47.786067 8145 mini_ranger.cc:193] Using host: 127.7.244.84
I20260812 00:35:47.806723 8145 mini_ranger.cc:227] Ranger admin URL: http://127.7.244.84:52759
Aug 12, 2026 12:35:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 12:35:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 12:35:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 12:35:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 12:35:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 12:35:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 12:35:48 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 00:35:49.236351 8145 mini_ranger.cc:160] Time spent starting Ranger: real 9.551s user 0.003s sys 0.064s
Aug 12, 2026 12:36:09 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 12:36:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 12:36:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 12:36:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 12:36:10 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 00:36:10.844897 8145 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.244.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.244.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/rangerkms_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/rangerkms_127.7.244.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.244.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.244.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.244.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/HTTP_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.244.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/HTTP_127.7.244.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 00:36:11 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494971, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 00:36:11.080968 8145 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 00:36:11.110950 8145 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 00:36:12.044124 8145 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 00:36:12.044293 8145 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms
I20260812 00:36:12.044437 8145 mini_ranger_kms.cc:227] Using postgres at 127.7.244.84:60841
2026-08-12 00:36:12,529 [I] DB FLAVOR :POSTGRES
2026-08-12 00:36:12,530 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 00:36:12,530 [I] Checking connection
2026-08-12 00:36:12,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 00:36:12,795 [I] connection success
2026-08-12 00:36:12,796 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 00:36:12,796 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 00:36:12,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 00:36:13,072 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 00:36:13,073 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 00:36:13,073 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 00:36:13,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:60841/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 00:36:13,345 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 00:36:13.820941 8145 mini_ranger_kms.cc:310] Created kms service
I20260812 00:36:14.042502 8145 mini_ranger_kms.cc:326] Created kudu user
I20260812 00:36:14.087981 8145 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 00:36:14 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494971, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:36:14.548292 8145 mini_ranger_kms.cc:384] Added ranger policy
I20260812 00:36:14.549649 8145 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 00:36:14.549731 8145 mini_ranger_kms.cc:240] Using host: 127.7.244.84
I20260812 00:36:14.582226 8145 mini_ranger_kms.cc:276] Ranger KMS PID: 31188
I20260812 00:36:14.582446 8145 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.244.84:59533
Aug 12, 2026 12:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 12:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 12:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 12:36:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 12:36:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 12:36:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 00:36:17 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786494977, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.244.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:36:17.425673 8145 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.345s user 0.157s sys 0.376s
I20260812 00:36:17.426086 8145 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 00:36:17.426340 8145 mini_ranger_kms.cc:401] 127.7.244.84:59533/kms/v1/keys
Aug 12 00:36:17 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494977, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.244.84@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
Aug 12 00:36:17 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494977, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.244.84@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
Aug 12 00:36:20 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:36:20.485390 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:40791
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:40791
--ranger_config_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:59533/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:36:20.970630 31299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:36:20.971174 31299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:36:20.971357 31299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:36:20.979996 31299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:36:20.980212 31299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:36:20.980321 31299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:36:20.980417 31299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:36:20.980476 31299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:36:20.993252 31299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.244.84:59533/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:40791
--ranger_config_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:40791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:36:20.994731 31299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:36:20.996343 31299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:36:21.012504 31306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:36:21.012355 31312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:36:21.012449 31307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:36:21.013345 31299 server_base.cc:1034] running on GCE node
I20260812 00:36:21.014364 31299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:36:21.016206 31299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:36:21.017499 31299 hybrid_clock.cc:648] HybridClock initialized: now 1786494981017359 us; error 58 us; skew 500 ppm
I20260812 00:36:21.018081 31299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:36:21 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494981, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:36:21.022320 31299 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:36:21.024770 31299 webserver.cc:460] Webserver started at http://127.7.244.126:35483/ using document root <none> and password file <none>
I20260812 00:36:21.025825 31299 fs_manager.cc:359] Metadata directory not provided
I20260812 00:36:21.026108 31299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:36:21.026595 31299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:36:21 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494981, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:36:21.102646 31299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "da567c30f3ed4ef1ae7fd2f136846e4b"
format_stamp: "Formatted at 2026-08-12 00:36:21 on dist-test-slave-gjw7"
server_key: "9e86bcb6fdd8f7029fca92a047eeea75"
server_key_iv: "5770bfe1b3e481e155a398b36d7bb91f"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:21.103706 31299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "da567c30f3ed4ef1ae7fd2f136846e4b"
format_stamp: "Formatted at 2026-08-12 00:36:21 on dist-test-slave-gjw7"
server_key: "9e86bcb6fdd8f7029fca92a047eeea75"
server_key_iv: "5770bfe1b3e481e155a398b36d7bb91f"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:21.110841 31299 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260812 00:36:21.133630 31326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:21.135706 31299 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:36:21.135973 31299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "da567c30f3ed4ef1ae7fd2f136846e4b"
format_stamp: "Formatted at 2026-08-12 00:36:21 on dist-test-slave-gjw7"
server_key: "9e86bcb6fdd8f7029fca92a047eeea75"
server_key_iv: "5770bfe1b3e481e155a398b36d7bb91f"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:21.136276 31299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:36:21.177866 31299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:36:21.183488 31299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:36:21.183902 31299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:36:21.214923 31299 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:40791
I20260812 00:36:21.215000 31445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:40791 every 8 connection(s)
I20260812 00:36:21.216732 31299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:36:21.221855 31448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:36:21.226328 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 31299
I20260812 00:36:21.226523 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:36:21.227260 8145 external_mini_cluster.cc:1428] Setting key b4ac969cd7f2dd28b5e0b88a6dc4c05f
I20260812 00:36:21.238579 31448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b: Bootstrap starting.
I20260812 00:36:21.242751 31448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b: Neither blocks nor log segments found. Creating new log.
I20260812 00:36:21.244011 31448 log.cc:824] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:36:21.247684 31448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b: No bootstrap required, opened a new log
Aug 12 00:36:21 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:36:21.258647 31453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:21.229597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.7.244.126:40791)
0812 00:36:21.230690 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:21.230707 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:21.230762 (+ 55us) server_negotiation.cc:407] Connection header received
0812 00:36:21.232998 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:21.233036 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:21.233819 (+ 783us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:21.234373 (+ 554us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:36:21.235587 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:21.237695 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:21.239132 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:21.240276 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:21.240508 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:21.249500 (+ 8992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:36:21.249513 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:36:21.249534 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:36:21.249582 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:36:21.253431 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:21.254589 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:21.254598 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:21.254607 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:21.254815 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:21.255367 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:21.255370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:21.255373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:21.256079 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:36:21.256355 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:21.256963 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 00:36:21.257448 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":569,"threads_started":1}
I20260812 00:36:21.258886 31448 raft_consensus.cc:348] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da567c30f3ed4ef1ae7fd2f136846e4b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 40791 } }
I20260812 00:36:21.259418 31448 raft_consensus.cc:374] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:36:21.259564 31448 raft_consensus.cc:729] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da567c30f3ed4ef1ae7fd2f136846e4b, State: Initialized, Role: FOLLOWER
I20260812 00:36:21.260334 31448 consensus_queue.cc:260] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da567c30f3ed4ef1ae7fd2f136846e4b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 40791 } }
I20260812 00:36:21.260739 31448 raft_consensus.cc:388] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:36:21.261006 31448 raft_consensus.cc:482] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:36:21.261399 31448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:36:21.264252 31448 raft_consensus.cc:504] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da567c30f3ed4ef1ae7fd2f136846e4b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 40791 } }
I20260812 00:36:21.264814 31448 leader_election.cc:302] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da567c30f3ed4ef1ae7fd2f136846e4b; no voters:
I20260812 00:36:21.265669 31448 leader_election.cc:288] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:36:21.265872 31455 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:36:21.267257 31455 raft_consensus.cc:686] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [term 1 LEADER]: Becoming Leader. State: Replica: da567c30f3ed4ef1ae7fd2f136846e4b, State: Running, Role: LEADER
I20260812 00:36:21.267939 31455 consensus_queue.cc:237] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da567c30f3ed4ef1ae7fd2f136846e4b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 40791 } }
I20260812 00:36:21.268580 31448 sys_catalog.cc:564] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:36:21.271750 31463 sys_catalog.cc:455] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da567c30f3ed4ef1ae7fd2f136846e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da567c30f3ed4ef1ae7fd2f136846e4b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 40791 } } }
I20260812 00:36:21.272185 31463 sys_catalog.cc:458] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [sys.catalog]: This master's current role is: LEADER
I20260812 00:36:21.273514 31464 sys_catalog.cc:455] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader da567c30f3ed4ef1ae7fd2f136846e4b. Latest consensus state: current_term: 1 leader_uuid: "da567c30f3ed4ef1ae7fd2f136846e4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da567c30f3ed4ef1ae7fd2f136846e4b" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 40791 } } }
I20260812 00:36:21.274446 31464 sys_catalog.cc:458] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b [sys.catalog]: This master's current role is: LEADER
I20260812 00:36:21.275525 31476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:36:21.283739 31476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:36:21.285207 31448 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 00:36:21.311424 31476 catalog_manager.cc:1347] Generated new cluster ID: aa775b7e706941c7942910ecfafe15ca
I20260812 00:36:21.311661 31476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:36:21.333809 31476 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:36:21.336151 31476 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:36:21.358139 31476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b: Generated new TSK 0
I20260812 00:36:21.359194 31476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:36:22.198868 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:59533/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:40791
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:36:22 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786494982, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 00:36:22.783273 31546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:36:22.783769 31546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:36:22.783953 31546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:36:22.796103 31546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:36:22.796284 31546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:36:22.796499 31546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:36:22.813431 31546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.244.84:59533/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:40791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:36:22.815398 31546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:36:22.817494 31546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:36:22.850298 31574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:36:22.850422 31572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:36:22.852094 31577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:36:22.852226 31546 server_base.cc:1034] running on GCE node
I20260812 00:36:22.852993 31546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:36:22.854142 31546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:36:22.855413 31546 hybrid_clock.cc:648] HybridClock initialized: now 1786494982855350 us; error 55 us; skew 500 ppm
I20260812 00:36:22.855796 31546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:36:22 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494982, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:36:22.859884 31546 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:36:22.862067 31546 webserver.cc:460] Webserver started at http://127.7.244.65:44335/ using document root <none> and password file <none>
I20260812 00:36:22.862943 31546 fs_manager.cc:359] Metadata directory not provided
I20260812 00:36:22.863194 31546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:36:22.863580 31546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:36:22 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494982, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:36:22.883396 31546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "72c85895d3d54da5a11740fafab53e8b"
format_stamp: "Formatted at 2026-08-12 00:36:22 on dist-test-slave-gjw7"
server_key: "0160ecf3a3f356d267e6bbfdbb5f9ac3"
server_key_iv: "31c85873a4dbe535fe2d1bd5e8b26a07"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:22.884487 31546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "72c85895d3d54da5a11740fafab53e8b"
format_stamp: "Formatted at 2026-08-12 00:36:22 on dist-test-slave-gjw7"
server_key: "0160ecf3a3f356d267e6bbfdbb5f9ac3"
server_key_iv: "31c85873a4dbe535fe2d1bd5e8b26a07"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:22.891069 31546 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 00:36:22.917582 31588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:22.922348 31546 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260812 00:36:22.922719 31546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "72c85895d3d54da5a11740fafab53e8b"
format_stamp: "Formatted at 2026-08-12 00:36:22 on dist-test-slave-gjw7"
server_key: "0160ecf3a3f356d267e6bbfdbb5f9ac3"
server_key_iv: "31c85873a4dbe535fe2d1bd5e8b26a07"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:22.923247 31546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:36:22.958479 31546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:36:22.964470 31546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:36:22.964828 31546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:36:22.966070 31546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:36:22.968072 31546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:36:22.968183 31546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:22.968431 31546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:36:22.968528 31546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:36:23 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494982, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:36:23.103925 31546 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:46277
I20260812 00:36:23.104144 31826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:46277 every 8 connection(s)
I20260812 00:36:23.106151 31546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:36:23.111742 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 31546
I20260812 00:36:23.111987 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:36:23.112897 8145 external_mini_cluster.cc:1428] Setting key 2b4ac6d989d97cf84dcc91d79175b0e9
Aug 12 00:36:23 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494982, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:36:23.176796 31836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:23.110729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:40791 (local address 127.7.244.65:56841)
0812 00:36:23.149733 (+ 39004us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:23.149810 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 00:36:23.152505 (+ 2695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:36:23.153936 (+ 1431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:36:23.153950 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:36:23.154682 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:36:23.155656 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:23.155687 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:23.157356 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:23.157363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:23.160185 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:23.160211 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:23.162877 (+ 2666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:23.162888 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:23.163207 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:23.164107 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:36:23.164139 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:36:23.167417 (+ 3278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:36:23.171618 (+ 4201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:36:23.171630 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:36:23.171647 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:36:23.172186 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:36:23.172692 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:36:23.172697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:36:23.172700 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:36:23.172963 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:36:23.173672 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:36:23.173682 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:36:23.174190 (+ 508us) client_negotiation.cc:769] Sending connection context
0812 00:36:23.174536 (+ 346us) client_negotiation.cc:240] Negotiation successful
0812 00:36:23.175172 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38623,"thread_start_us":38243,"threads_started":1}
I20260812 00:36:23.179003 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:23.110072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:56841 (local address 127.7.244.126:40791)
0812 00:36:23.148669 (+ 38597us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:23.148676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:23.151616 (+ 2940us) server_negotiation.cc:407] Connection header received
0812 00:36:23.153145 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:23.153151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:23.153255 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:23.153382 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:36:23.155957 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:23.157083 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:23.161094 (+ 4011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:23.162201 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:23.162357 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:23.167775 (+ 5418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:36:23.167783 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:36:23.167788 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:36:23.167851 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:36:23.171422 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:23.172349 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:23.172354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:23.172356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:23.172552 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:23.173104 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:23.173108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:23.173111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:23.173508 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:36:23.177971 (+ 4463us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:23.178166 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 00:36:23.178480 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38437,"thread_start_us":38250,"threads_started":1}
I20260812 00:36:23.181555 31832 heartbeater.cc:347] Connected to a master server at 127.7.244.126:40791
I20260812 00:36:23.182062 31832 heartbeater.cc:464] Registering TS with master...
I20260812 00:36:23.183091 31832 heartbeater.cc:511] Master 127.7.244.126:40791 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:36:23.186321 31361 ts_manager.cc:194] Registered new tserver with Master: 72c85895d3d54da5a11740fafab53e8b (127.7.244.65:46277)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
I20260812 00:36:23.188835 31361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:56841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
W20260812 00:36:23.232494 8145 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.244.66: real 0.113s user 0.002s sys 0.021s
I20260812 00:36:23.233606 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:59533/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:40791
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:36:23 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494982, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
W20260812 00:36:23.839949 31844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:36:23.840392 31844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:36:23.840653 31844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:36:23.849094 31844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:36:23.849337 31844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:36:23.849548 31844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:36:23.862655 31844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.244.84:59533/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:40791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:36:23.864421 31844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:36:23.866107 31844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:36:23.883040 31875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:36:23.883390 31878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:36:23.885263 31882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:36:23.886173 31844 server_base.cc:1034] running on GCE node
I20260812 00:36:23.887081 31844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:36:23.888679 31844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:36:23.889950 31844 hybrid_clock.cc:648] HybridClock initialized: now 1786494983889876 us; error 74 us; skew 500 ppm
I20260812 00:36:23.890352 31844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:36:23 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:36:23.895319 31844 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:36:23.897095 31844 webserver.cc:460] Webserver started at http://127.7.244.66:38351/ using document root <none> and password file <none>
I20260812 00:36:23.898025 31844 fs_manager.cc:359] Metadata directory not provided
I20260812 00:36:23.898151 31844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:36:23.898464 31844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:36:23 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:36:23.919333 31844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "cd236bfaffdf4797858c4a58c1a4d788"
format_stamp: "Formatted at 2026-08-12 00:36:23 on dist-test-slave-gjw7"
server_key: "fe48ecfaab2afd4931dac05bb74e1234"
server_key_iv: "9d5fddc83d823a39adfa0c83d837245e"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:23.920966 31844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "cd236bfaffdf4797858c4a58c1a4d788"
format_stamp: "Formatted at 2026-08-12 00:36:23 on dist-test-slave-gjw7"
server_key: "fe48ecfaab2afd4931dac05bb74e1234"
server_key_iv: "9d5fddc83d823a39adfa0c83d837245e"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:23.930120 31844 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.005s
I20260812 00:36:23.949652 31893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:23.951300 31844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:36:23.951517 31844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "cd236bfaffdf4797858c4a58c1a4d788"
format_stamp: "Formatted at 2026-08-12 00:36:23 on dist-test-slave-gjw7"
server_key: "fe48ecfaab2afd4931dac05bb74e1234"
server_key_iv: "9d5fddc83d823a39adfa0c83d837245e"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:23.951756 31844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:36:23.982551 31844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:36:23.987392 31844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:36:23.988027 31844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:36:23.989892 31844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:36:23.992292 31844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:36:23.992388 31844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:23.992518 31844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:36:23.992568 31844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:24.051479 31844 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:38291
I20260812 00:36:24.051508 32119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:38291 every 8 connection(s)
I20260812 00:36:24.052879 31844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:36:24.058140 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 31844
I20260812 00:36:24.058357 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:36:24.058985 8145 external_mini_cluster.cc:1428] Setting key d462c6d08100d7631bf0ea719d64381e
Aug 12 00:36:24 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494983, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:36:24.080632 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.055737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:35643 (local address 127.7.244.126:40791)
0812 00:36:24.056390 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:24.056397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:24.057757 (+ 1360us) server_negotiation.cc:407] Connection header received
0812 00:36:24.060538 (+ 2781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:24.060542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:24.060666 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:24.060772 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:36:24.063052 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.064225 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.066768 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.067511 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.067626 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.072616 (+ 4990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:36:24.072627 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:36:24.072633 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:36:24.072743 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:36:24.076643 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:24.077738 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:24.077745 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:24.077749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:24.077946 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:24.078465 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:24.078471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:24.078475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:24.078865 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:36:24.079023 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:24.079848 (+ 825us) server_negotiation.cc:299] Negotiation successful
0812 00:36:24.080089 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":340,"threads_started":1}
I20260812 00:36:24.081771 32126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.056462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:40791 (local address 127.7.244.66:35643)
0812 00:36:24.057574 (+ 1112us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:24.057616 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:36:24.060216 (+ 2600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:36:24.061098 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:36:24.061107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:36:24.061789 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:36:24.062809 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.062837 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.064433 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.064440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.066568 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.066591 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.067795 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.067803 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.068141 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.068979 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:36:24.069011 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:36:24.072188 (+ 3177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:36:24.076880 (+ 4692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:36:24.076890 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:36:24.076907 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:36:24.077556 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:36:24.078071 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:36:24.078075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:36:24.078077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:36:24.078337 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:36:24.079028 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:36:24.079035 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:36:24.079425 (+ 390us) client_negotiation.cc:769] Sending connection context
0812 00:36:24.079682 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 00:36:24.080123 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"thread_start_us":528,"threads_started":1}
I20260812 00:36:24.084066 32120 heartbeater.cc:347] Connected to a master server at 127.7.244.126:40791
I20260812 00:36:24.084450 32120 heartbeater.cc:464] Registering TS with master...
I20260812 00:36:24.085505 32120 heartbeater.cc:511] Master 127.7.244.126:40791 requested a full tablet report, sending...
I20260812 00:36:24.087724 31361 ts_manager.cc:194] Registered new tserver with Master: cd236bfaffdf4797858c4a58c1a4d788 (127.7.244.66:38291)
I20260812 00:36:24.088730 31361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:35643
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:36:24.128413 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:59533/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:40791
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:36:24.193936 31832 heartbeater.cc:503] Master 127.7.244.126:40791 was elected leader, sending a full tablet report...
W20260812 00:36:24.581247 32129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:36:24.581696 32129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:36:24.581821 32129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:36:24.591377 32129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:36:24.591609 32129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:36:24.591877 32129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:36:24.606369 32129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.244.84:59533/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:40791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:36:24.607944 32129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:36:24.609625 32129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:36:24.625206 32141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:36:24.626266 32146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:36:24.626327 32142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:36:24.628008 32129 server_base.cc:1034] running on GCE node
I20260812 00:36:24.628736 32129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:36:24.630084 32129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:36:24.631325 32129 hybrid_clock.cc:648] HybridClock initialized: now 1786494984631253 us; error 76 us; skew 500 ppm
I20260812 00:36:24.631698 32129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:36:24 dist-test-slave-gjw7 krb5kdc[30369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:36:24.635463 32129 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:36:24.637351 32129 webserver.cc:460] Webserver started at http://127.7.244.67:44145/ using document root <none> and password file <none>
I20260812 00:36:24.638175 32129 fs_manager.cc:359] Metadata directory not provided
I20260812 00:36:24.638321 32129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:36:24.638805 32129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:36:24 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:36:24.657591 32129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "d70005c454894f01a759fea77533f4a0"
format_stamp: "Formatted at 2026-08-12 00:36:24 on dist-test-slave-gjw7"
server_key: "949ae2cff02e6c5cea7bab4742c84654"
server_key_iv: "365ff4e48d32999b42255f3c2ba94772"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:24.658857 32129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "d70005c454894f01a759fea77533f4a0"
format_stamp: "Formatted at 2026-08-12 00:36:24 on dist-test-slave-gjw7"
server_key: "949ae2cff02e6c5cea7bab4742c84654"
server_key_iv: "365ff4e48d32999b42255f3c2ba94772"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:24.665845 32129 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:36:24.682643 32158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:24.684438 32129 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260812 00:36:24.684645 32129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "d70005c454894f01a759fea77533f4a0"
format_stamp: "Formatted at 2026-08-12 00:36:24 on dist-test-slave-gjw7"
server_key: "949ae2cff02e6c5cea7bab4742c84654"
server_key_iv: "365ff4e48d32999b42255f3c2ba94772"
server_key_version: "kuduclusterkey@0"
I20260812 00:36:24.684904 32129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:36:24.702194 32129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:36:24.706449 32129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:36:24.706758 32129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:36:24.707826 32129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:36:24.709823 32129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:36:24.709996 32129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:24.710216 32129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:36:24.710286 32129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:36:24.753656 32129 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:46659
I20260812 00:36:24.753665 32407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:46659 every 8 connection(s)
I20260812 00:36:24.755159 32129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:36:24.760356 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 32129
I20260812 00:36:24.760576 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:36:24.761129 8145 external_mini_cluster.cc:1428] Setting key beb0c8e5da044676c051816d68e26c7e
Aug 12 00:36:24 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:36:24.778976 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.758006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:49485 (local address 127.7.244.126:40791)
0812 00:36:24.758557 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:24.758562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:24.759602 (+ 1040us) server_negotiation.cc:407] Connection header received
0812 00:36:24.762286 (+ 2684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:24.762293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:24.762369 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:24.762471 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:36:24.764680 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.765667 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.767721 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.768185 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.768293 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.772153 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:36:24.772159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:36:24.772162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:36:24.772203 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:36:24.775283 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:24.776370 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:24.776375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:24.776377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:24.776590 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:24.777229 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:24.777234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:24.777237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:24.777671 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:36:24.777797 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:24.778523 (+ 726us) server_negotiation.cc:299] Negotiation successful
0812 00:36:24.778715 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":242,"threads_started":1}
I20260812 00:36:24.780128 32414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.758574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:40791 (local address 127.7.244.67:49485)
0812 00:36:24.759432 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:24.759472 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 00:36:24.761950 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:36:24.762740 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:36:24.762753 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:36:24.763557 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:36:24.764508 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.764532 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.765809 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.765813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.767457 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.767517 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.768490 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.768498 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.768729 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.769432 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:36:24.769463 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:36:24.771844 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:36:24.775487 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:36:24.775498 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:36:24.775534 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:36:24.776232 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:36:24.776710 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:36:24.776716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:36:24.776719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:36:24.777099 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:36:24.777804 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:36:24.777811 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:36:24.778195 (+ 384us) client_negotiation.cc:769] Sending connection context
0812 00:36:24.778449 (+ 254us) client_negotiation.cc:240] Negotiation successful
0812 00:36:24.778938 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":384,"threads_started":1}
I20260812 00:36:24.781653 32408 heartbeater.cc:347] Connected to a master server at 127.7.244.126:40791
I20260812 00:36:24.782016 32408 heartbeater.cc:464] Registering TS with master...
I20260812 00:36:24.782827 32408 heartbeater.cc:511] Master 127.7.244.126:40791 requested a full tablet report, sending...
I20260812 00:36:24.784595 31361 ts_manager.cc:194] Registered new tserver with Master: d70005c454894f01a759fea77533f4a0 (127.7.244.67:46659)
I20260812 00:36:24.785665 31361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:49485
I20260812 00:36:24.790441 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:36:24.810346 32413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.795741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.7.244.126:40791)
0812 00:36:24.795928 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:24.795933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:24.796360 (+ 427us) server_negotiation.cc:407] Connection header received
0812 00:36:24.796528 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:24.796532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:24.796617 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:24.796706 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:36:24.798318 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.799109 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.800798 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.801262 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.801435 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.804133 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:36:24.804139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:36:24.804143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:36:24.804184 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:36:24.807205 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:24.808052 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:24.808059 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:24.808061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:24.808223 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:24.808679 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:24.808684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:24.808686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:24.809004 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:36:24.809261 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:24.809737 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 00:36:24.809990 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:36:24.814754 31361 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:36:24.823029 31361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:36:24.864712 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.838727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.7.244.66:38291)
0812 00:36:24.852201 (+ 13474us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:24.852214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:24.853492 (+ 1278us) server_negotiation.cc:407] Connection header received
0812 00:36:24.853782 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:24.853792 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:24.854129 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:24.854372 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:36:24.856223 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.858781 (+ 2558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.861907 (+ 3126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.863212 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.863351 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.863593 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:24.863991 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 00:36:24.864248 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13186,"thread_start_us":17968,"threads_started":1}
I20260812 00:36:24.864760 32441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.838880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:38291 (local address 127.0.0.1:58132)
0812 00:36:24.853340 (+ 14460us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:24.853367 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 00:36:24.853590 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:36:24.855256 (+ 1666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:36:24.855266 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:36:24.855306 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:36:24.856009 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.856027 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.858912 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.858916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.861670 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.861695 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.863521 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.863529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.863725 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.863770 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 00:36:24.863905 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 00:36:24.864293 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14258,"mutex_wait_us":31,"thread_start_us":11228,"threads_started":1}
I20260812 00:36:24.865952 32446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.838256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:46659 (local address 127.0.0.1:58724)
0812 00:36:24.849508 (+ 11252us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:24.849574 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 00:36:24.849846 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:36:24.857965 (+ 8119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:36:24.857970 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:36:24.858002 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:36:24.858498 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.858514 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.860266 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.860276 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.862522 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.862539 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.865065 (+ 2526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.865073 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.865267 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.865325 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 00:36:24.865415 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:36:24.865563 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10846,"mutex_wait_us":4,"thread_start_us":15441,"threads_started":1}
I20260812 00:36:24.866096 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.838533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58724 (local address 127.7.244.67:46659)
0812 00:36:24.856741 (+ 18208us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:24.856750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:24.856788 (+ 38us) server_negotiation.cc:407] Connection header received
0812 00:36:24.856901 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:24.856910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:24.857750 (+ 840us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:24.857989 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:36:24.858705 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.860105 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.862688 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.864217 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.864350 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.864593 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:24.865496 (+ 903us) server_negotiation.cc:299] Negotiation successful
0812 00:36:24.865673 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17996,"thread_start_us":17903,"threads_started":1}
I20260812 00:36:24.868531 32450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.851165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46830)
0812 00:36:24.858285 (+ 7120us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:24.858304 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:36:24.858465 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:36:24.859239 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:36:24.859244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:36:24.859271 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:36:24.859830 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.859844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.861699 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.861705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.864401 (+ 2696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:24.864421 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.866730 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.866737 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:24.867868 (+ 1131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.867907 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 00:36:24.868011 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 00:36:24.868160 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6977,"thread_start_us":7193,"threads_started":1}
I20260812 00:36:24.868927 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:24.851739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.7.244.65:46277)
0812 00:36:24.857659 (+ 5920us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:24.857669 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:24.858603 (+ 934us) server_negotiation.cc:407] Connection header received
0812 00:36:24.858719 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:24.858726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:24.859000 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:24.859214 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:36:24.860035 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.861474 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.864605 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:24.866076 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:24.866322 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:24.866550 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:24.868095 (+ 1545us) server_negotiation.cc:299] Negotiation successful
0812 00:36:24.868339 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5666,"thread_start_us":5371,"threads_started":1}
I20260812 00:36:24.869817 31990 tablet_service.cc:1467] Processing CreateTablet for tablet 819062a7b1e34cc884c3a5d8736f0e67 (DEFAULT_TABLE table=test-table [id=dfc934039fb9412c92560d3e75ba1900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:36:24.870519 32267 tablet_service.cc:1467] Processing CreateTablet for tablet 819062a7b1e34cc884c3a5d8736f0e67 (DEFAULT_TABLE table=test-table [id=dfc934039fb9412c92560d3e75ba1900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:36:24.871491 31990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 819062a7b1e34cc884c3a5d8736f0e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:36:24.872007 32267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 819062a7b1e34cc884c3a5d8736f0e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:36:24.873319 31698 tablet_service.cc:1467] Processing CreateTablet for tablet 819062a7b1e34cc884c3a5d8736f0e67 (DEFAULT_TABLE table=test-table [id=dfc934039fb9412c92560d3e75ba1900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:36:24.875741 31698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 819062a7b1e34cc884c3a5d8736f0e67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:36:24.888132 32452 tablet_bootstrap.cc:492] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Bootstrap starting.
I20260812 00:36:24.893656 32452 tablet_bootstrap.cc:654] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Neither blocks nor log segments found. Creating new log.
I20260812 00:36:24.894106 32457 tablet_bootstrap.cc:492] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: Bootstrap starting.
I20260812 00:36:24.894210 32455 tablet_bootstrap.cc:492] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Bootstrap starting.
I20260812 00:36:24.896076 32452 log.cc:824] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:36:24.900043 32455 tablet_bootstrap.cc:654] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Neither blocks nor log segments found. Creating new log.
I20260812 00:36:24.900689 32457 tablet_bootstrap.cc:654] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: Neither blocks nor log segments found. Creating new log.
I20260812 00:36:24.902345 32455 log.cc:824] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Log is configured to *not* fsync() on all Append() calls
I20260812 00:36:24.903080 32457 log.cc:824] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:36:24.904978 32452 tablet_bootstrap.cc:492] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: No bootstrap required, opened a new log
I20260812 00:36:24.905381 32452 ts_tablet_manager.cc:1397] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.009s
I20260812 00:36:24.905411 32455 tablet_bootstrap.cc:492] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: No bootstrap required, opened a new log
I20260812 00:36:24.905706 32455 ts_tablet_manager.cc:1397] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260812 00:36:24.908777 32457 tablet_bootstrap.cc:492] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: No bootstrap required, opened a new log
I20260812 00:36:24.909114 32457 ts_tablet_manager.cc:1397] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260812 00:36:24.914170 32455 raft_consensus.cc:348] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:24.914615 32455 raft_consensus.cc:374] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:36:24.914718 32455 raft_consensus.cc:729] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd236bfaffdf4797858c4a58c1a4d788, State: Initialized, Role: FOLLOWER
I20260812 00:36:24.915395 32455 consensus_queue.cc:260] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:24.916627 32120 heartbeater.cc:503] Master 127.7.244.126:40791 was elected leader, sending a full tablet report...
I20260812 00:36:24.917171 32455 ts_tablet_manager.cc:1428] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260812 00:36:24.918867 32452 raft_consensus.cc:348] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:24.919265 32452 raft_consensus.cc:374] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:36:24.919466 32452 raft_consensus.cc:729] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d70005c454894f01a759fea77533f4a0, State: Initialized, Role: FOLLOWER
I20260812 00:36:24.919963 32457 raft_consensus.cc:348] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:24.920158 32452 consensus_queue.cc:260] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:24.920614 32457 raft_consensus.cc:374] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:36:24.920790 32457 raft_consensus.cc:729] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c85895d3d54da5a11740fafab53e8b, State: Initialized, Role: FOLLOWER
I20260812 00:36:24.921624 32457 consensus_queue.cc:260] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:24.923468 32408 heartbeater.cc:503] Master 127.7.244.126:40791 was elected leader, sending a full tablet report...
I20260812 00:36:24.924141 32452 ts_tablet_manager.cc:1428] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Time spent starting tablet: real 0.019s user 0.011s sys 0.005s
I20260812 00:36:24.930822 32457 ts_tablet_manager.cc:1428] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: Time spent starting tablet: real 0.022s user 0.018s sys 0.003s
W20260812 00:36:25.007045 32411 tablet.cc:2369] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:36:25.055191 32121 tablet.cc:2369] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:36:25.092526 32466 raft_consensus.cc:482] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:36:25.093101 32466 raft_consensus.cc:504] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:25.095434 32466 leader_election.cc:288] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d70005c454894f01a759fea77533f4a0 (127.7.244.67:46659), 72c85895d3d54da5a11740fafab53e8b (127.7.244.65:46277)
I20260812 00:36:25.103413 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.095977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:35645 (local address 127.7.244.65:46277)
0812 00:36:25.096261 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:25.096267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:25.097149 (+ 882us) server_negotiation.cc:407] Connection header received
0812 00:36:25.097507 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:25.097511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:25.097573 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:25.097646 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:36:25.098251 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.099339 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.101601 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.102514 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.102675 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:25.102756 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:25.103034 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 00:36:25.103171 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 00:36:25.103485 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.095908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.7.244.66:35645)
0812 00:36:25.097044 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.097064 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:36:25.097329 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:36:25.097683 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:36:25.097688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:36:25.097716 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:36:25.098101 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:25.098113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.099514 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.099520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:25.101434 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:25.101450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.102736 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.102743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:25.102881 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:25.102906 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 00:36:25.102990 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 00:36:25.103131 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"thread_start_us":742,"threads_started":1}
I20260812 00:36:25.105149 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.096404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:46659 (local address 127.7.244.66:48003)
0812 00:36:25.097642 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.097658 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:36:25.097802 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:36:25.098129 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:36:25.098134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:36:25.098158 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:36:25.098719 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:25.098730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.100224 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.100229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:25.102592 (+ 2363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:36:25.102609 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.104301 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.104307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:36:25.104532 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:25.104558 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 00:36:25.104615 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 00:36:25.104712 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"thread_start_us":859,"threads_started":1}
I20260812 00:36:25.104693 31735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "819062a7b1e34cc884c3a5d8736f0e67" candidate_uuid: "cd236bfaffdf4797858c4a58c1a4d788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c85895d3d54da5a11740fafab53e8b" is_pre_election: true
I20260812 00:36:25.105901 31735 raft_consensus.cc:2454] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd236bfaffdf4797858c4a58c1a4d788 in term 0.
I20260812 00:36:25.106197 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.096093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:48003 (local address 127.7.244.67:46659)
0812 00:36:25.096305 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:25.096311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:25.097766 (+ 1455us) server_negotiation.cc:407] Connection header received
0812 00:36:25.097948 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:25.097953 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:25.098014 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:25.098091 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:36:25.098927 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.100064 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.102804 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.104041 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.105715 (+ 1674us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:25.105776 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:25.105822 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 00:36:25.105924 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 00:36:25.106910 31896 leader_election.cc:302] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72c85895d3d54da5a11740fafab53e8b, cd236bfaffdf4797858c4a58c1a4d788; no voters:
I20260812 00:36:25.107095 32307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "819062a7b1e34cc884c3a5d8736f0e67" candidate_uuid: "cd236bfaffdf4797858c4a58c1a4d788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d70005c454894f01a759fea77533f4a0" is_pre_election: true
I20260812 00:36:25.107416 32466 raft_consensus.cc:2781] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:36:25.107537 32466 raft_consensus.cc:482] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:36:25.107599 32466 raft_consensus.cc:3037] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:36:25.107597 32307 raft_consensus.cc:2454] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd236bfaffdf4797858c4a58c1a4d788 in term 0.
I20260812 00:36:25.110816 32466 raft_consensus.cc:504] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:25.111496 32466 leader_election.cc:288] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [CANDIDATE]: Term 1 election: Requested vote from peers d70005c454894f01a759fea77533f4a0 (127.7.244.67:46659), 72c85895d3d54da5a11740fafab53e8b (127.7.244.65:46277)
I20260812 00:36:25.112147 31735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "819062a7b1e34cc884c3a5d8736f0e67" candidate_uuid: "cd236bfaffdf4797858c4a58c1a4d788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72c85895d3d54da5a11740fafab53e8b"
I20260812 00:36:25.112161 32307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "819062a7b1e34cc884c3a5d8736f0e67" candidate_uuid: "cd236bfaffdf4797858c4a58c1a4d788" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d70005c454894f01a759fea77533f4a0"
I20260812 00:36:25.112360 31735 raft_consensus.cc:3037] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:36:25.112375 32307 raft_consensus.cc:3037] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:36:25.114950 31735 raft_consensus.cc:2454] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd236bfaffdf4797858c4a58c1a4d788 in term 1.
I20260812 00:36:25.114964 32307 raft_consensus.cc:2454] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd236bfaffdf4797858c4a58c1a4d788 in term 1.
I20260812 00:36:25.115707 31904 leader_election.cc:302] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd236bfaffdf4797858c4a58c1a4d788, d70005c454894f01a759fea77533f4a0; no voters:
I20260812 00:36:25.116106 32466 raft_consensus.cc:2781] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:36:25.116603 32466 raft_consensus.cc:686] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 1 LEADER]: Becoming Leader. State: Replica: cd236bfaffdf4797858c4a58c1a4d788, State: Running, Role: LEADER
I20260812 00:36:25.117112 32466 consensus_queue.cc:237] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } }
I20260812 00:36:25.122624 31361 catalog_manager.cc:5495] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd236bfaffdf4797858c4a58c1a4d788 (127.7.244.66). New cstate: current_term: 1 leader_uuid: "cd236bfaffdf4797858c4a58c1a4d788" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd236bfaffdf4797858c4a58c1a4d788" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 38291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 } health_report { overall_health: UNKNOWN } } }
W20260812 00:36:25.162919 31833 tablet.cc:2369] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:36:25.169039 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.162731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58146 (local address 127.7.244.66:38291)
0812 00:36:25.162966 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:25.162971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:25.163043 (+ 72us) server_negotiation.cc:407] Connection header received
0812 00:36:25.163321 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:25.163326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:25.163428 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:25.163558 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:36:25.164094 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.164940 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.166647 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.167074 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.167182 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:25.167833 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:36:25.168406 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:36:25.168558 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:25.168673 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 00:36:25.168772 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:36:25.179390 31735 raft_consensus.cc:1261] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 1 FOLLOWER]: Refusing update from remote peer cd236bfaffdf4797858c4a58c1a4d788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:36:25.179396 32307 raft_consensus.cc:1261] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 1 FOLLOWER]: Refusing update from remote peer cd236bfaffdf4797858c4a58c1a4d788: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:36:25.180536 32476 consensus_queue.cc:1035] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d70005c454894f01a759fea77533f4a0" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 46659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:36:25.181126 32466 consensus_queue.cc:1035] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72c85895d3d54da5a11740fafab53e8b" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 46277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:36:25.213214 31360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:36:25.213640 31360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:36:25.218743 31360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da567c30f3ed4ef1ae7fd2f136846e4b: Sending DeleteTablet for 3 replicas of tablet 819062a7b1e34cc884c3a5d8736f0e67
I20260812 00:36:25.220293 32267 tablet_service.cc:1514] Processing DeleteTablet for tablet 819062a7b1e34cc884c3a5d8736f0e67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:36:25 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:58724
I20260812 00:36:25.220568 31990 tablet_service.cc:1514] Processing DeleteTablet for tablet 819062a7b1e34cc884c3a5d8736f0e67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:36:25 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:58132
I20260812 00:36:25.221616 31698 tablet_service.cc:1514] Processing DeleteTablet for tablet 819062a7b1e34cc884c3a5d8736f0e67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:36:25 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:46830
I20260812 00:36:25.224941 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 31546
I20260812 00:36:25.235488 32506 tablet_replica.cc:330] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: stopping tablet replica
I20260812 00:36:25.236069 32506 raft_consensus.cc:2229] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:36:25.236682 32506 raft_consensus.cc:2258] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:36:25.243296 32507 tablet_replica.cc:330] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: stopping tablet replica
I20260812 00:36:25.244021 32507 raft_consensus.cc:2229] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 1 LEADER]: Raft consensus shutting down.
Aug 12 00:36:25 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:36:25.244805 32507 raft_consensus.cc:2258] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:36:25.246346 32506 ts_tablet_manager.cc:1905] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:36:25.247195 32507 ts_tablet_manager.cc:1905] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:36:25.252601 32510 tablet_replica.cc:330] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: stopping tablet replica
I20260812 00:36:25.253134 32510 raft_consensus.cc:2229] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:36:25.253803 32510 raft_consensus.cc:2258] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:36:25.255146 32507 ts_tablet_manager.cc:1918] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:36:25.255326 32507 log.cc:1196] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/819062a7b1e34cc884c3a5d8736f0e67
I20260812 00:36:25.257656 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.226998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46834 (local address 127.7.244.65:46277)
0812 00:36:25.227254 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:25.227263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:25.229703 (+ 2440us) server_negotiation.cc:407] Connection header received
0812 00:36:25.230329 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:25.230338 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:25.230491 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:25.230605 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:36:25.234886 (+ 4281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.236290 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.238732 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.239470 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.239642 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:25.246835 (+ 7193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:36:25.246859 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:36:25.246878 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:36:25.246943 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:36:25.252324 (+ 5381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:25.254499 (+ 2175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:25.254511 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:25.254514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:25.254720 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:25.255305 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:25.255312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:25.255316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:25.255939 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:36:25.256275 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:25.256962 (+ 687us) server_negotiation.cc:299] Negotiation successful
0812 00:36:25.257230 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 00:36:25.258229 32510 ts_tablet_manager.cc:1905] T 819062a7b1e34cc884c3a5d8736f0e67 P 72c85895d3d54da5a11740fafab53e8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:36:25.259732 31822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:36:25.264149 32507 ts_tablet_manager.cc:1939] T 819062a7b1e34cc884c3a5d8736f0e67 P cd236bfaffdf4797858c4a58c1a4d788: Deleting consensus metadata
I20260812 00:36:25.266522 31327 catalog_manager.cc:4841] TS cd236bfaffdf4797858c4a58c1a4d788 (127.7.244.66:38291): tablet 819062a7b1e34cc884c3a5d8736f0e67 (table test-table [id=dfc934039fb9412c92560d3e75ba1900]) successfully deleted
I20260812 00:36:25.266526 32506 ts_tablet_manager.cc:1918] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:36:25.266855 32506 log.cc:1196] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/819062a7b1e34cc884c3a5d8736f0e67
I20260812 00:36:25.267491 32506 ts_tablet_manager.cc:1939] T 819062a7b1e34cc884c3a5d8736f0e67 P d70005c454894f01a759fea77533f4a0: Deleting consensus metadata
I20260812 00:36:25.269493 31334 catalog_manager.cc:4841] TS d70005c454894f01a759fea77533f4a0 (127.7.244.67:46659): tablet 819062a7b1e34cc884c3a5d8736f0e67 (table test-table [id=dfc934039fb9412c92560d3e75ba1900]) successfully deleted
I20260812 00:36:25.358781 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 31546
I20260812 00:36:25.380782 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 31844
W20260812 00:36:25.380686 31327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.244.65:46277 (error 108)
I20260812 00:36:25.383232 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.382358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46842)
0812 00:36:25.382845 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.383004 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":242,"threads_started":1}
W20260812 00:36:25.383579 31327 catalog_manager.cc:4584] TS 72c85895d3d54da5a11740fafab53e8b (127.7.244.65:46277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 819062a7b1e34cc884c3a5d8736f0e67: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Aug 12 00:36:25 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:36:25.400637 32445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.382141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.7.244.66:38291)
0812 00:36:25.382301 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:25.382306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:25.382323 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:36:25.382393 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:25.382397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:25.382460 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:25.382572 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:36:25.384565 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.385766 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.387186 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.387661 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.387790 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:25.392602 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:36:25.392627 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:36:25.392651 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:36:25.392709 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:36:25.397215 (+ 4506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:25.398137 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:25.398153 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:25.398156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:25.398364 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:25.398844 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:25.398849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:25.398852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:25.399380 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:36:25.399736 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:25.400026 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 00:36:25.400285 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:36:25.401551 32111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:36:25.437599 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.436933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46844)
0812 00:36:25.437215 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.437439 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 00:36:25.438818 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.438384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46852)
0812 00:36:25.438589 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.438663 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 00:36:25.499562 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.499047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46862)
0812 00:36:25.499277 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.499404 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 00:36:25.500720 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.500318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46864)
0812 00:36:25.500483 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.500561 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 00:36:25.517158 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 31844
I20260812 00:36:25.545430 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 32129
Aug 12 00:36:25 dist-test-slave-gjw7 krb5kdc[30369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:36:25.561854 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.546218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.7.244.67:46659)
0812 00:36:25.546450 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 00:36:25.546456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:36:25.546478 (+ 22us) server_negotiation.cc:407] Connection header received
0812 00:36:25.546636 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:36:25.546640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:36:25.546709 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:36:25.546796 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:36:25.548237 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.548981 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.550534 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:36:25.550935 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:36:25.551054 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:36:25.555722 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:36:25.555742 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:36:25.555761 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:36:25.555806 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:36:25.558788 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:25.559592 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:25.559601 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:25.559603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:25.559752 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:36:25.560151 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:36:25.560155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:36:25.560157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:36:25.560578 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:36:25.560885 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0812 00:36:25.561210 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 00:36:25.561467 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 00:36:25.562659 32399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:36:25.581647 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.581063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46868)
0812 00:36:25.581344 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.581443 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 00:36:25.582854 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.582410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46874)
0812 00:36:25.582615 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.582697 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260812 00:36:25.653411 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 32129
I20260812 00:36:25.678922 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 31299
I20260812 00:36:25.680317 31429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:36:25.794503 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:36:25.792996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:46277 (local address 127.0.0.1:46878)
0812 00:36:25.794074 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0812 00:36:25.794254 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:46277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":797}
I20260812 00:36:25.794998 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 31299
I20260812 00:36:25.829317 8145 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 00:36:26.177608 8145 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 00:36:26.177868 8145 mini_ranger.cc:66] Stopping Ranger...
I20260812 00:36:26.524245 8145 mini_ranger.cc:68] Stopped Ranger
2026-08-12 00:36:26.524 UTC [30435] LOG: received smart shutdown request
2026-08-12 00:36:26.529 UTC [30435] LOG: background worker "logical replication launcher" (PID 30453) exited with exit code 1
2026-08-12 00:36:26.529 UTC [30443] LOG: shutting down
2026-08-12 00:36:26.706 UTC [30435] LOG: database system is shut down
2026-08-12T00:36:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52662 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:36:26 dist-test-slave-gjw7 krb5kdc[32546](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:36:26 dist-test-slave-gjw7 krb5kdc[32546](info): set up 2 sockets
Aug 12 00:36:26 dist-test-slave-gjw7 krb5kdc[32546](info): commencing operation
krb5kdc: starting...
W20260812 00:36:28.147498 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:36:28 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786494988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:36:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:36:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.244.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.244.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/rangeradmin_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/rangeradmin_127.7.244.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.244.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.244.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/rangerlookup_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/rangerlookup_127.7.244.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.244.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.244.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/HTTP_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/HTTP_127.7.244.84@KRBTEST.COM.keytab.
I20260812 00:36:28.549250 8145 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOGbh_J/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/postgres -l logfile start
2026-08-12 00:36:31.141 UTC [32610] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 00:36:31.141 UTC [32610] LOG: listening on IPv4 address "127.7.244.84", port 37403
2026-08-12 00:36:31.145 UTC [32610] LOG: listening on Unix socket "/tmp/.s.PGSQL.37403"
2026-08-12 00:36:31.168 UTC [32614] LOG: database system was shut down at 2026-08-12 00:36:29 UTC
2026-08-12 00:36:31.173 UTC [32610] LOG: database system is ready to accept connections
I20260812 00:36:32.350349 8145 mini_postgres.cc:96] Postgres bound to 37403
2026-08-12 00:36:32.372 UTC [32630] FATAL: database "slave" does not exist
127.7.244.84:37403 - accepting connections
I20260812 00:36:32.373499 8145 mini_ranger.cc:161] Starting Ranger...
I20260812 00:36:32.406001 8145 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 00:36:32.938436 8145 mini_ranger.cc:87] Created ranger Postgres database
I20260812 00:36:32.938566 8145 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-admin
2026-08-12 00:36:33,254 [I] DB FLAVOR :POSTGRES
2026-08-12 00:36:33,254 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 00:36:33,254 [I] Checking connection..
2026-08-12 00:36:33,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 00:36:33,540 [I] Checking connection passed.
2026-08-12 00:36:33,541 [I] --------- Verifying version history table ---------
2026-08-12 00:36:33,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:36:33,824 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 00:36:33,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:36:34,080 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 00:36:34,080 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 00:36:34,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 00:36:34.340 UTC [32760] WARNING: there is no transaction in progress
2026-08-12 00:36:34,349 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 00:36:34,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 00:36:34,621 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 00:36:34,621 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 00:36:34,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 00:36:34,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 00:36:35,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjw7.c.gcp-upstream.internal','N') ;"
2026-08-12 00:36:35,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 00:36:35,726 [I] Table x_portal_user does not exist in database ranger
2026-08-12 00:36:35,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 00:36:36,001 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 00:36:36,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 00:36:36,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 00:36:36,569 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 00:36:36,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 00:36:37.642 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.661 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.680 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.697 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.714 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.734 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.799 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.807 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.824 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.835 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.847 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.863 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.881 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.891 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.900 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:37.908 UTC [503] WARNING: there is no transaction in progress
2026-08-12 00:36:38,709 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 00:36:38,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 00:36:38,990 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 00:36:38,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 00:36:39,241 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 00:36:39,241 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 00:36:39,517 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 00:36:39,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 00:36:39,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 00:36:40,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjw7.c.gcp-upstream.internal';"
2026-08-12 00:36:40,300 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 00:36:40,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 00:36:40,580 [I] DB_PATCHES have already been applied
I20260812 00:36:40.586473 8145 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOGbh_J/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 00:36:40.586571 8145 mini_ranger.cc:193] Using host: 127.7.244.84
I20260812 00:36:40.606550 8145 mini_ranger.cc:227] Ranger admin URL: http://127.7.244.84:54541
Aug 12, 2026 12:36:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 12:36:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 12:36:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 12:36:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 12:36:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 12:36:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 12:36:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 00:36:42.021793 8145 mini_ranger.cc:160] Time spent starting Ranger: real 9.648s user 0.004s sys 0.059s
Aug 12, 2026 12:37:03 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 12:37:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 12:37:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 12:37:03 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 12:37:04 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 00:37:04.581840 8145 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.244.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.244.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/rangerkms_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.244.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/rangerkms_127.7.244.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.244.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.244.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.244.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/HTTP_127.7.244.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.244.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/HTTP_127.7.244.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 00:37:04 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495024, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 00:37:04.788755 8145 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 00:37:04.814627 8145 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 00:37:05.693173 8145 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 00:37:05.693332 8145 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms
I20260812 00:37:05.693388 8145 mini_ranger_kms.cc:227] Using postgres at 127.7.244.84:37403
2026-08-12 00:37:06,055 [I] DB FLAVOR :POSTGRES
2026-08-12 00:37:06,055 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 00:37:06,056 [I] Checking connection
2026-08-12 00:37:06,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 00:37:06,318 [I] connection success
2026-08-12 00:37:06,318 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 00:37:06,318 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 00:37:06,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 00:37:06,576 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 00:37:06,577 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 00:37:06,577 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 00:37:06,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.244.84:37403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 00:37:06,888 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 00:37:07.444180 8145 mini_ranger_kms.cc:310] Created kms service
I20260812 00:37:07.641650 8145 mini_ranger_kms.cc:326] Created kudu user
I20260812 00:37:07.689826 8145 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 00:37:07 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495024, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:08.103807 8145 mini_ranger_kms.cc:384] Added ranger policy
I20260812 00:37:08.104188 8145 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOGbh_J/build/asan/bin/postgresql.jar:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOGbh_J/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 00:37:08.104254 8145 mini_ranger_kms.cc:240] Using host: 127.7.244.84
I20260812 00:37:08.126348 8145 mini_ranger_kms.cc:276] Ranger KMS PID: 922
I20260812 00:37:08.126493 8145 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.244.84:46469
Aug 12, 2026 12:37:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 12:37:09 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 12:37:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 12:37:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 12:37:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 12:37:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 00:37:10 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786495030, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.244.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 00:37:10 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495030, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.244.84@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:10.913833 8145 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.125s user 0.074s sys 0.324s
I20260812 00:37:10.914084 8145 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 00:37:10.914248 8145 mini_ranger_kms.cc:401] 127.7.244.84:46469/kms/v1/keys
Aug 12 00:37:11 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495030, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.244.84@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
Aug 12 00:37:13 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:37:13.518422 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:37481
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:37481
--ranger_config_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:46469/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:13.872005 1025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:13.872344 1025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:13.872437 1025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:13.880277 1025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:13.880434 1025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:13.880517 1025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:13.880594 1025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:13.880661 1025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:13.893007 1025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.244.84:46469/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:37481
--ranger_config_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:37481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:13.894459 1025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:13.895955 1025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:13.909122 1037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:13.909046 1043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:13.909071 1046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:13.909863 1025 server_base.cc:1034] running on GCE node
I20260812 00:37:13.910744 1025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:13.912312 1025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:13.913527 1025 hybrid_clock.cc:648] HybridClock initialized: now 1786495033913478 us; error 48 us; skew 500 ppm
I20260812 00:37:13.913900 1025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:13 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:13.920961 1025 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:37:13.922946 1025 webserver.cc:460] Webserver started at http://127.7.244.126:45773/ using document root <none> and password file <none>
I20260812 00:37:13.923748 1025 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:13.923890 1025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:13.924227 1025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:37:13 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495033, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:13.996004 1025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "850c6aa02cb54b04be606cda5cdaba26"
format_stamp: "Formatted at 2026-08-12 00:37:13 on dist-test-slave-gjw7"
server_key: "198ae179e6519d585da2f7ebf52f0ca3"
server_key_iv: "4b5c15c69c02af7c2a6f723f51e0f8dd"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:13.997032 1025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "850c6aa02cb54b04be606cda5cdaba26"
format_stamp: "Formatted at 2026-08-12 00:37:13 on dist-test-slave-gjw7"
server_key: "198ae179e6519d585da2f7ebf52f0ca3"
server_key_iv: "4b5c15c69c02af7c2a6f723f51e0f8dd"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:14.003180 1025 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260812 00:37:14.025417 1055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:14.027369 1025 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260812 00:37:14.027639 1025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "850c6aa02cb54b04be606cda5cdaba26"
format_stamp: "Formatted at 2026-08-12 00:37:13 on dist-test-slave-gjw7"
server_key: "198ae179e6519d585da2f7ebf52f0ca3"
server_key_iv: "4b5c15c69c02af7c2a6f723f51e0f8dd"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:14.027951 1025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:14.053910 1025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:14.059535 1025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:14.059913 1025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:14.080955 1025 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:37481
I20260812 00:37:14.080910 1177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:37481 every 8 connection(s)
I20260812 00:37:14.082858 1025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:37:14.087318 1178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:14.089251 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 1025
I20260812 00:37:14.089533 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:37:14.090396 8145 external_mini_cluster.cc:1428] Setting key 33a0cb53cc7bb7727788ddc1df052689
I20260812 00:37:14.104144 1178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Bootstrap starting.
I20260812 00:37:14.109017 1178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:14.110373 1178 log.cc:824] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:14.113957 1178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: No bootstrap required, opened a new log
Aug 12 00:37:14 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:14.123692 1178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } }
I20260812 00:37:14.124203 1178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:14.124349 1178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 850c6aa02cb54b04be606cda5cdaba26, State: Initialized, Role: FOLLOWER
I20260812 00:37:14.125098 1178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } }
I20260812 00:37:14.125455 1178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:37:14.125624 1178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:37:14.125836 1178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:14.129149 1178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } }
I20260812 00:37:14.129824 1178 leader_election.cc:302] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 850c6aa02cb54b04be606cda5cdaba26; no voters:
I20260812 00:37:14.131882 1178 leader_election.cc:288] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:37:14.132077 1191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:37:14.134796 1182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:14.092841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.7.244.126:37481)
0812 00:37:14.107729 (+ 14888us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:14.107748 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:14.107817 (+ 69us) server_negotiation.cc:407] Connection header received
0812 00:37:14.110576 (+ 2759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:14.110616 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:14.111224 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:14.111727 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:14.113416 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:14.115431 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:14.117563 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:14.118726 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:14.118960 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:14.124175 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:14.124192 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:14.124217 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:14.124276 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:14.128410 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:14.129514 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:14.129526 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:14.129537 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:14.129839 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:14.130405 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:14.130410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:14.130413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:14.131244 (+ 831us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:14.131564 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:14.132191 (+ 627us) server_negotiation.cc:299] Negotiation successful
0812 00:37:14.132812 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14576,"thread_start_us":14290,"threads_started":1}
I20260812 00:37:14.136298 1191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 1 LEADER]: Becoming Leader. State: Replica: 850c6aa02cb54b04be606cda5cdaba26, State: Running, Role: LEADER
I20260812 00:37:14.137053 1191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } }
I20260812 00:37:14.141196 1178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:14.145859 1199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 850c6aa02cb54b04be606cda5cdaba26. Latest consensus state: current_term: 1 leader_uuid: "850c6aa02cb54b04be606cda5cdaba26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } } }
I20260812 00:37:14.146446 1199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:14.145898 1203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "850c6aa02cb54b04be606cda5cdaba26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } } }
I20260812 00:37:14.146984 1203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:14.147516 1212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:37:14.159317 1212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:37:14.185026 1212 catalog_manager.cc:1347] Generated new cluster ID: ae426148b613477fab29df638684f22d
I20260812 00:37:14.185189 1212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:37:14.224730 1212 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:37:14.227191 1212 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:37:14.238451 1178 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 00:37:14.261149 1212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Generated new TSK 0
I20260812 00:37:14.262781 1212 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:37:15.098415 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:46469/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:37481
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:37:15 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786495035, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 00:37:15.589174 1289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:15.589692 1289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:15.589867 1289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:15.601903 1289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:15.602101 1289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:15.602314 1289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:37:15.618898 1289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.244.84:46469/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:15.621114 1289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:15.623246 1289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:15.645428 1304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:15.650743 1307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:15.651563 1311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:15.651908 1289 server_base.cc:1034] running on GCE node
I20260812 00:37:15.652769 1289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:15.654196 1289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:15.655488 1289 hybrid_clock.cc:648] HybridClock initialized: now 1786495035655407 us; error 65 us; skew 500 ppm
I20260812 00:37:15.656049 1289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:15 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495035, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:15.660830 1289 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:37:15.662868 1289 webserver.cc:460] Webserver started at http://127.7.244.65:33685/ using document root <none> and password file <none>
I20260812 00:37:15.663694 1289 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:15.663836 1289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:15.664166 1289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:37:15 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495035, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:15.683102 1289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "7dbba47ad58d4cfd8c70cf8de98f6712"
format_stamp: "Formatted at 2026-08-12 00:37:15 on dist-test-slave-gjw7"
server_key: "60c49c267f033e2440b09b94e85e39a5"
server_key_iv: "dd324781d018d9712a9f8ef870ecad86"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:15.684058 1289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "7dbba47ad58d4cfd8c70cf8de98f6712"
format_stamp: "Formatted at 2026-08-12 00:37:15 on dist-test-slave-gjw7"
server_key: "60c49c267f033e2440b09b94e85e39a5"
server_key_iv: "dd324781d018d9712a9f8ef870ecad86"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:15.691237 1289 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260812 00:37:15.717746 1325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:15.721473 1289 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.006s sys 0.002s
I20260812 00:37:15.721838 1289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "7dbba47ad58d4cfd8c70cf8de98f6712"
format_stamp: "Formatted at 2026-08-12 00:37:15 on dist-test-slave-gjw7"
server_key: "60c49c267f033e2440b09b94e85e39a5"
server_key_iv: "dd324781d018d9712a9f8ef870ecad86"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:15.722271 1289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:15.766538 1289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:15.772053 1289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:15.772378 1289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:15.773591 1289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:15.775231 1289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:15.775322 1289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:15.775481 1289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:15.775547 1289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:15.838195 1289 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:36221
I20260812 00:37:15.838207 1559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:36221 every 8 connection(s)
I20260812 00:37:15.839807 1289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:37:15.843132 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 1289
I20260812 00:37:15.843372 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:37:15.844275 8145 external_mini_cluster.cc:1428] Setting key 4aeeb60c5529140e6a9ab1bec274138f
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Aug 12 00:37:15 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495035, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Aug 12 00:37:15 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495035, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:15.903510 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:15.859787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:40309 (local address 127.7.244.126:37481)
0812 00:37:15.864965 (+ 5178us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:15.864970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:15.872646 (+ 7676us) server_negotiation.cc:407] Connection header received
0812 00:37:15.879974 (+ 7328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:15.879982 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:15.880113 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:15.880202 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:15.882757 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:15.883879 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:15.886707 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:15.887327 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:15.887449 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:15.895184 (+ 7735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:15.895194 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:15.895198 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:15.895261 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:15.898892 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:15.900142 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:15.900149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:15.900153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:15.900354 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:15.901020 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:15.901026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:15.901030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:15.901471 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:15.901614 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:15.902848 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0812 00:37:15.903118 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5018,"thread_start_us":4888,"threads_started":1}
I20260812 00:37:15.904982 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:15.854396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:37481 (local address 127.7.244.65:40309)
0812 00:37:15.872390 (+ 17994us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:15.872472 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:15.879512 (+ 7040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:15.880869 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:15.880884 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:15.881575 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:15.882496 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:15.882525 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:15.884138 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:15.884143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:15.886184 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:15.886203 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:15.887833 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:15.887840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:15.888155 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:15.889025 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:15.889061 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:15.894782 (+ 5721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:15.899161 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:15.899174 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:15.899199 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:15.899945 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:15.900534 (+ 589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:15.900538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:15.900542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:15.900851 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:15.901731 (+ 880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:15.901741 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:15.902280 (+ 539us) client_negotiation.cc:769] Sending connection context
0812 00:37:15.902649 (+ 369us) client_negotiation.cc:240] Negotiation successful
0812 00:37:15.903298 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17567,"thread_start_us":18137,"threads_started":1}
I20260812 00:37:15.907372 1562 heartbeater.cc:347] Connected to a master server at 127.7.244.126:37481
I20260812 00:37:15.907761 1562 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:15.908679 1562 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
I20260812 00:37:15.911468 1101 ts_manager.cc:194] Registered new tserver with Master: 7dbba47ad58d4cfd8c70cf8de98f6712 (127.7.244.65:36221)
I20260812 00:37:15.913667 1101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:40309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:37:15.943279 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:46469/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:37481
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:37:16 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495035, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
W20260812 00:37:16.445488 1572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:16.446056 1572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:16.446254 1572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:16.457386 1572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:16.457664 1572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:16.457895 1572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:37:16.474124 1572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.244.84:46469/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:16.476274 1572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:16.478250 1572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:16.496834 1595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:16.498291 1591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:16.505882 1572 server_base.cc:1034] running on GCE node
W20260812 00:37:16.507088 1598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:16.508325 1572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:16.516307 1572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:16.517916 1572 hybrid_clock.cc:648] HybridClock initialized: now 1786495036517820 us; error 83 us; skew 500 ppm
I20260812 00:37:16.518425 1572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:16 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495036, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:16.523274 1572 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:37:16.525593 1572 webserver.cc:460] Webserver started at http://127.7.244.66:39969/ using document root <none> and password file <none>
I20260812 00:37:16.526643 1572 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:16.526795 1572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:16.527172 1572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:37:16 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495036, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:16.559315 1572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "b8300a32716e4804a04fe82e5e0844f3"
format_stamp: "Formatted at 2026-08-12 00:37:16 on dist-test-slave-gjw7"
server_key: "706379220d83c47a24ec1ac4be3b2114"
server_key_iv: "c159b5613151a7928cd3a2c46a9ab409"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:16.560647 1572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "b8300a32716e4804a04fe82e5e0844f3"
format_stamp: "Formatted at 2026-08-12 00:37:16 on dist-test-slave-gjw7"
server_key: "706379220d83c47a24ec1ac4be3b2114"
server_key_iv: "c159b5613151a7928cd3a2c46a9ab409"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:16.568990 1572 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260812 00:37:16.614158 1622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:16.623528 1572 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.009s sys 0.000s
I20260812 00:37:16.623847 1572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "b8300a32716e4804a04fe82e5e0844f3"
format_stamp: "Formatted at 2026-08-12 00:37:16 on dist-test-slave-gjw7"
server_key: "706379220d83c47a24ec1ac4be3b2114"
server_key_iv: "c159b5613151a7928cd3a2c46a9ab409"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:16.624146 1572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:16.693677 1572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:16.717337 1572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:16.717804 1572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:16.729164 1572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:16.734221 1572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:16.734457 1572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 00:37:16.734673 1572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:16.734829 1572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:16.808900 1572 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:35119
I20260812 00:37:16.808956 1855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:35119 every 8 connection(s)
I20260812 00:37:16.810321 1572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:37:16.811858 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 1572
I20260812 00:37:16.812038 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:37:16.812625 8145 external_mini_cluster.cc:1428] Setting key 5a49530827a9ee500ec630ee94110b3e
Aug 12 00:37:16 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495036, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:16.838027 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:16.813137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:36973 (local address 127.7.244.126:37481)
0812 00:37:16.813745 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:16.813751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:16.815056 (+ 1305us) server_negotiation.cc:407] Connection header received
0812 00:37:16.817783 (+ 2727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:16.817788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:16.817902 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:16.818005 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:16.820106 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:16.821173 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:16.823749 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:16.824321 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:16.824465 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:16.829119 (+ 4654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:16.829127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:16.829132 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:16.829184 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:16.832950 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:16.834315 (+ 1365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:16.834321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:16.834325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:16.834514 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:16.835178 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:16.835184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:16.835188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:16.835654 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:16.835838 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:16.837360 (+ 1522us) server_negotiation.cc:299] Negotiation successful
0812 00:37:16.837644 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":1493,"threads_started":1}
I20260812 00:37:16.840790 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:16.813866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:37481 (local address 127.7.244.66:36973)
0812 00:37:16.814882 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:16.814947 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:16.817309 (+ 2362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:16.818336 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:16.818349 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:16.818980 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:16.819848 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:16.819879 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:16.821416 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:16.821422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:16.823532 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:16.823553 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:16.824603 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:16.824611 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:16.824931 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:16.825783 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:16.825816 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:16.828750 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:16.833223 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:16.833234 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:16.833257 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:16.834132 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:16.834656 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:16.834660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:16.834664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:16.835003 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:16.835831 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:16.835841 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:16.836389 (+ 548us) client_negotiation.cc:769] Sending connection context
0812 00:37:16.838637 (+ 2248us) client_negotiation.cc:240] Negotiation successful
0812 00:37:16.839260 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":379,"threads_started":1}
I20260812 00:37:16.843147 1856 heartbeater.cc:347] Connected to a master server at 127.7.244.126:37481
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
I20260812 00:37:16.843544 1856 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:16.844460 1856 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
I20260812 00:37:16.846494 1101 ts_manager.cc:194] Registered new tserver with Master: b8300a32716e4804a04fe82e5e0844f3 (127.7.244.66:35119)
I20260812 00:37:16.847612 1101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:36973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:37:16.880231 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:46469/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:37481
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:37:16.918098 1562 heartbeater.cc:503] Master 127.7.244.126:37481 was elected leader, sending a full tablet report...
W20260812 00:37:17.229650 1867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:17.229993 1867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:17.230145 1867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:17.237921 1867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:17.238058 1867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:17.238214 1867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:37:17.248576 1867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.244.84:46469/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:17.250100 1867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:17.251539 1867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:17.265491 1881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:17.265565 1886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:17.265527 1876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:17.266454 1867 server_base.cc:1034] running on GCE node
I20260812 00:37:17.267112 1867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:17.268245 1867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:17.269448 1867 hybrid_clock.cc:648] HybridClock initialized: now 1786495037269386 us; error 58 us; skew 500 ppm
I20260812 00:37:17.269795 1867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:17 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495037, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:17.273411 1867 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:37:17.275044 1867 webserver.cc:460] Webserver started at http://127.7.244.67:35161/ using document root <none> and password file <none>
I20260812 00:37:17.275821 1867 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:17.275972 1867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:17.276290 1867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 00:37:17 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495037, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:17.293836 1867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "08cd113a026d44769b722d413d70fc1e"
format_stamp: "Formatted at 2026-08-12 00:37:17 on dist-test-slave-gjw7"
server_key: "a06d04fec171ee2e1ef143a4035d53b6"
server_key_iv: "8a777df28f1a9a6f71222d6668d53327"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:17.294847 1867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "08cd113a026d44769b722d413d70fc1e"
format_stamp: "Formatted at 2026-08-12 00:37:17 on dist-test-slave-gjw7"
server_key: "a06d04fec171ee2e1ef143a4035d53b6"
server_key_iv: "8a777df28f1a9a6f71222d6668d53327"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:17.301024 1867 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:37:17.318697 1900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:17.320380 1867 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:37:17.320616 1867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "08cd113a026d44769b722d413d70fc1e"
format_stamp: "Formatted at 2026-08-12 00:37:17 on dist-test-slave-gjw7"
server_key: "a06d04fec171ee2e1ef143a4035d53b6"
server_key_iv: "8a777df28f1a9a6f71222d6668d53327"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:17.320885 1867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:17.349485 1867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:17.353730 1867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:17.354045 1867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:17.355110 1867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:17.356638 1867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:17.356716 1867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:17.356835 1867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:17.356889 1867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:17.389916 1867 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:45251
I20260812 00:37:17.389958 2125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:45251 every 8 connection(s)
I20260812 00:37:17.391275 1867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:37:17.399441 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 1867
I20260812 00:37:17.399613 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:37:17.400087 8145 external_mini_cluster.cc:1428] Setting key 8a472ed4eb5bc40434db698e2977799c
Aug 12 00:37:17 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495037, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:17.412233 2131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:17.394071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:55251 (local address 127.7.244.126:37481)
0812 00:37:17.394600 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:17.394605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:17.395631 (+ 1026us) server_negotiation.cc:407] Connection header received
0812 00:37:17.397512 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:17.397517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:17.397605 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:17.397702 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:17.399261 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.400085 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.402447 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.403018 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.403151 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:17.406607 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:17.406612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:17.406616 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:17.406657 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:17.409179 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:17.410044 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:17.410048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:17.410050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:17.410200 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:17.410605 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:17.410608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:17.410610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:17.410891 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:17.411023 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:17.411722 (+ 699us) server_negotiation.cc:299] Negotiation successful
0812 00:37:17.411948 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":244,"threads_started":1}
I20260812 00:37:17.413236 2135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:17.394680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:37481 (local address 127.7.244.67:55251)
0812 00:37:17.395455 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:17.395497 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:17.397173 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:17.397976 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:17.397985 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:17.398448 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:17.399067 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:17.399092 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.400279 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.400287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:17.402277 (+ 1990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:17.402298 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.403269 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.403275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:17.403512 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:17.404239 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:17.404264 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:17.406360 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:17.409400 (+ 3040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:17.409408 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:17.409424 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:17.409925 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:17.410288 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:17.410291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:17.410293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:17.410513 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:17.410998 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:17.411004 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:17.411370 (+ 366us) client_negotiation.cc:769] Sending connection context
0812 00:37:17.411613 (+ 243us) client_negotiation.cc:240] Negotiation successful
0812 00:37:17.412059 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":315,"threads_started":1}
I20260812 00:37:17.414958 2126 heartbeater.cc:347] Connected to a master server at 127.7.244.126:37481
I20260812 00:37:17.415334 2126 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:17.416342 2126 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
I20260812 00:37:17.417894 1101 ts_manager.cc:194] Registered new tserver with Master: 08cd113a026d44769b722d413d70fc1e (127.7.244.67:45251)
I20260812 00:37:17.418725 1101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:55251
I20260812 00:37:17.429453 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:37:17.431248 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 1289
Aug 12 00:37:17 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:37:17.449324 2139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:17.432221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50948 (local address 127.7.244.65:36221)
0812 00:37:17.432860 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:17.432868 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:17.432896 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:37:17.432959 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:17.432965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:17.433199 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:17.433420 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:17.434992 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.436085 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.437572 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.438226 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.438343 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:17.443111 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:17.443130 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:17.443149 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:17.443200 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:17.446358 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:17.447249 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:17.447258 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:17.447260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:17.447481 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:17.447848 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:17.447851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:17.447853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:17.448224 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:17.448508 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:17.448742 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 00:37:17.448979 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":319,"threads_started":1}
I20260812 00:37:17.450774 1550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:17.534900 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 1289
I20260812 00:37:17.563371 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 1572
Aug 12 00:37:17 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:37:17.582474 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:17.564327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39204 (local address 127.7.244.66:35119)
0812 00:37:17.564874 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:17.564881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:17.564911 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:37:17.564980 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:17.564986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:17.565197 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:17.565408 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:17.566963 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.568087 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.569783 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.570429 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.570544 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:17.575477 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:17.575504 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:17.575523 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:17.575588 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:17.578954 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:17.579917 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:17.579926 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:17.579928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:17.580122 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:17.580649 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:17.580654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:17.580656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:17.581100 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:17.581441 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:17.581803 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 00:37:17.582057 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":278,"threads_started":1}
I20260812 00:37:17.583853 1847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:17.666944 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 1572
I20260812 00:37:17.694636 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 1867
Aug 12 00:37:17 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:37:17.711562 2146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:17.695758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.7.244.67:45251)
0812 00:37:17.696486 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:17.696494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:17.696525 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:37:17.696598 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:17.696604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:17.696836 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:17.696976 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:17.698388 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.699443 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.700731 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:17.701353 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:17.701480 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:17.705915 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:17.705929 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:17.705947 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:17.705986 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:17.708718 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:17.709574 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:17.709583 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:17.709585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:17.709780 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:17.710111 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:17.710114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:17.710116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:17.710483 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:17.710731 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:17.710997 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 00:37:17.711214 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":399,"threads_started":1}
I20260812 00:37:17.712829 2117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:17.796257 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 1867
I20260812 00:37:17.817607 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 1025
I20260812 00:37:17.819243 1169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:17.929993 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 1025
I20260812 00:37:17.966811 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:37481
--webserver_interface=127.7.244.126
--webserver_port=45773
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:37481
--ranger_config_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:46469/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.244.84:46469/kms with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:18.328765 2154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:18.329119 2154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:18.329237 2154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:18.336957 2154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:18.337095 2154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:18.337143 2154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:18.337220 2154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:18.337284 2154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:18.347929 2154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.244.84:46469/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:37481
--ranger_config_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:37481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=45773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:18.349483 2154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:18.350987 2154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:18.364193 2176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:18.364193 2170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:18.364223 2172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:18.364924 2154 server_base.cc:1034] running on GCE node
I20260812 00:37:18.365881 2154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:18.367488 2154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:18.368741 2154 hybrid_clock.cc:648] HybridClock initialized: now 1786495038368683 us; error 48 us; skew 500 ppm
I20260812 00:37:18.369146 2154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:18 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:18.372786 2154 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:37:18.374625 2154 webserver.cc:460] Webserver started at http://127.7.244.126:45773/ using document root <none> and password file <none>
I20260812 00:37:18.375483 2154 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:18.375628 2154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 00:37:18 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495038, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:18.454686 2154 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:37:18.458249 2189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:18.460005 2154 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 00:37:18.460249 2154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "850c6aa02cb54b04be606cda5cdaba26"
format_stamp: "Formatted at 2026-08-12 00:37:13 on dist-test-slave-gjw7"
server_key: "198ae179e6519d585da2f7ebf52f0ca3"
server_key_iv: "4b5c15c69c02af7c2a6f723f51e0f8dd"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:18.461380 2154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:18.479583 2154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:18.483860 2154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:18.484167 2154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:18.503638 2154 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:37481
I20260812 00:37:18.503662 2287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:37481 every 8 connection(s)
I20260812 00:37:18.505470 2154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:37:18.507248 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 2154
I20260812 00:37:18.508111 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:46469/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:36221
--local_ip_for_outbound_sockets=127.7.244.65
--tserver_master_addrs=127.7.244.126:37481
--webserver_port=33685
--webserver_interface=127.7.244.65
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.244.84:46469/kms with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:37:18.513341 2288 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:37:18.516923 2288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Bootstrap starting.
I20260812 00:37:18.536170 2288 log.cc:824] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:18.546324 2288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:37:18.546919 2288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Bootstrap complete.
I20260812 00:37:18.556140 2288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } }
I20260812 00:37:18.557140 2288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 850c6aa02cb54b04be606cda5cdaba26, State: Initialized, Role: FOLLOWER
I20260812 00:37:18.557928 2288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } }
I20260812 00:37:18.558187 2288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:37:18.558316 2288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:37:18.558485 2288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:37:18.562628 2288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } }
I20260812 00:37:18.563216 2288 leader_election.cc:302] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 850c6aa02cb54b04be606cda5cdaba26; no voters:
I20260812 00:37:18.563906 2288 leader_election.cc:288] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 00:37:18.564059 2300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 00:37:18.565429 2300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [term 2 LEADER]: Becoming Leader. State: Replica: 850c6aa02cb54b04be606cda5cdaba26, State: Running, Role: LEADER
I20260812 00:37:18.566161 2300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } }
I20260812 00:37:18.566488 2288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:18.568802 2312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 850c6aa02cb54b04be606cda5cdaba26. Latest consensus state: current_term: 2 leader_uuid: "850c6aa02cb54b04be606cda5cdaba26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } } }
I20260812 00:37:18.569271 2312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:18.569926 2306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "850c6aa02cb54b04be606cda5cdaba26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850c6aa02cb54b04be606cda5cdaba26" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37481 } } }
I20260812 00:37:18.570235 2306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:18.578832 2320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:37:18.585602 2320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:37:18.587366 2320 catalog_manager.cc:1259] Loaded cluster ID: ae426148b613477fab29df638684f22d
I20260812 00:37:18.587584 2320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:37:18.592092 2320 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:37:18.594022 2320 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Loaded TSK: 0
I20260812 00:37:18.595891 2320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:37:18.599741 2288 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260812 00:37:18.996825 2293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:18.997471 2293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:18.997692 2293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:19.010205 2293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:19.010391 2293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:19.010663 2293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:37:19.027323 2293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.244.84:46469/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:36221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=33685
--tserver_master_addrs=127.7.244.126:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:19.029497 2293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:19.031574 2293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:19.063629 2377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:19.064985 2376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:19.070747 2293 server_base.cc:1034] running on GCE node
W20260812 00:37:19.071064 2379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:19.072113 2293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:19.073755 2293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:19.075116 2293 hybrid_clock.cc:648] HybridClock initialized: now 1786495039075032 us; error 75 us; skew 500 ppm
I20260812 00:37:19.075562 2293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:19 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:19.085642 2293 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:37:19.112074 2293 webserver.cc:460] Webserver started at http://127.7.244.65:33685/ using document root <none> and password file <none>
I20260812 00:37:19.113267 2293 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:19.113507 2293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 00:37:19 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:19.161509 2293 fs_manager.cc:709] Time spent opening directory manager: real 0.024s user 0.008s sys 0.009s
I20260812 00:37:19.167725 2390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:19.170171 2293 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.003s
I20260812 00:37:19.170421 2293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "7dbba47ad58d4cfd8c70cf8de98f6712"
format_stamp: "Formatted at 2026-08-12 00:37:15 on dist-test-slave-gjw7"
server_key: "60c49c267f033e2440b09b94e85e39a5"
server_key_iv: "dd324781d018d9712a9f8ef870ecad86"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:19.172032 2293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:19.198079 2293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:19.202837 2293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:19.203142 2293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:19.204296 2293 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:19.206888 2293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:19.207021 2293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.001s sys 0.000s
I20260812 00:37:19.207228 2293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:19.207389 2293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:19.262676 2293 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:36221
I20260812 00:37:19.262777 2625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:36221 every 8 connection(s)
I20260812 00:37:19.265090 2293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:37:19.274392 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 2293
I20260812 00:37:19.275070 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:46469/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:35119
--local_ip_for_outbound_sockets=127.7.244.66
--tserver_master_addrs=127.7.244.126:37481
--webserver_port=39969
--webserver_interface=127.7.244.66
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.244.84:46469/kms with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:37:19 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:19.312084 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:19.280037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:37481 (local address 127.7.244.65:45039)
0812 00:37:19.281108 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:19.281174 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:19.283875 (+ 2701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:19.288337 (+ 4462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:19.288351 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:19.289066 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:19.289987 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:19.290022 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:19.292443 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:19.292449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:19.294674 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:19.294699 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:19.296412 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:19.296423 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:19.296730 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:19.297592 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:19.297632 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:19.300674 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:19.306172 (+ 5498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:19.306185 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:19.306206 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:19.306963 (+ 757us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:19.307660 (+ 697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:19.307666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:19.307671 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:19.307987 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:19.309080 (+ 1093us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:19.309090 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:19.309578 (+ 488us) client_negotiation.cc:769] Sending connection context
0812 00:37:19.309889 (+ 311us) client_negotiation.cc:240] Negotiation successful
0812 00:37:19.310515 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":425,"threads_started":1}
I20260812 00:37:19.315482 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:19.278060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:45039 (local address 127.7.244.126:37481)
0812 00:37:19.286363 (+ 8303us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:19.286389 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:19.286452 (+ 63us) server_negotiation.cc:407] Connection header received
0812 00:37:19.286945 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:19.286970 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:19.287585 (+ 615us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:19.288127 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:19.290255 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:19.292159 (+ 1904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:19.294896 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:19.296097 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:19.296350 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:19.301134 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:19.301148 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:19.301175 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:19.301237 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:19.305898 (+ 4661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:19.307223 (+ 1325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:19.307235 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:19.307244 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:19.307507 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:19.308154 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:19.308160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:19.308164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:19.308905 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:19.313062 (+ 4157us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:19.313470 (+ 408us) server_negotiation.cc:299] Negotiation successful
0812 00:37:19.314004 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8011,"thread_start_us":7738,"threads_started":1}
I20260812 00:37:19.319962 2631 heartbeater.cc:347] Connected to a master server at 127.7.244.126:37481
I20260812 00:37:19.320333 2631 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:19.321278 2631 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
W20260812 00:37:19.325340 2631 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:37481 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 00:37:19.325592 2631 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:19.325893 2631 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
I20260812 00:37:19.327476 2631 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:19.327925 2631 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
W20260812 00:37:19.329247 2631 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:37:19.734860 2642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:19.735210 2642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:19.735355 2642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:19.744712 2642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:19.744884 2642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:19.745061 2642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:37:19.760337 2642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.244.84:46469/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:35119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=39969
--tserver_master_addrs=127.7.244.126:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:19.762408 2642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:19.764276 2642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:19.789664 2681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:19.790417 2675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:19.797724 2642 server_base.cc:1034] running on GCE node
W20260812 00:37:19.798291 2687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:19.799468 2642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:19.801003 2642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:19.802315 2642 hybrid_clock.cc:648] HybridClock initialized: now 1786495039802188 us; error 129 us; skew 500 ppm
I20260812 00:37:19.802807 2642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:19 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:19.807590 2642 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:37:19.815124 2642 webserver.cc:460] Webserver started at http://127.7.244.66:39969/ using document root <none> and password file <none>
I20260812 00:37:19.816306 2642 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:19.816558 2642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 00:37:19 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:19.860555 2642 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.011s sys 0.004s
I20260812 00:37:19.865018 2701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:19.867347 2642 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260812 00:37:19.867672 2642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "b8300a32716e4804a04fe82e5e0844f3"
format_stamp: "Formatted at 2026-08-12 00:37:16 on dist-test-slave-gjw7"
server_key: "706379220d83c47a24ec1ac4be3b2114"
server_key_iv: "c159b5613151a7928cd3a2c46a9ab409"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:19.869230 2642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 12 00:37:19 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:19.899873 2642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:19.909107 2642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:19.909593 2642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:19.911028 2642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:19.913458 2642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:19.913583 2642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:19.913712 2642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:19.913786 2642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:19.953868 2642 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:35119
I20260812 00:37:19.953925 2930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:35119 every 8 connection(s)
I20260812 00:37:19.956091 2642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:37:19.959585 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 2642
I20260812 00:37:19.960466 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.244.84:46469/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:45251
--local_ip_for_outbound_sockets=127.7.244.67
--tserver_master_addrs=127.7.244.126:37481
--webserver_port=35161
--webserver_interface=127.7.244.67
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:39891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.244.84:46469/kms with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:37:19 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:20.005935 2939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:19.976419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:43883 (local address 127.7.244.126:37481)
0812 00:37:19.977760 (+ 1341us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:19.977767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:19.981094 (+ 3327us) server_negotiation.cc:407] Connection header received
0812 00:37:19.983257 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:19.983265 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:19.983379 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:19.983529 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:19.985863 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:19.987067 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:19.989757 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:19.990376 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:19.990528 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:19.995359 (+ 4831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:19.995368 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:19.995373 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:19.995466 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:19.999154 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:20.000508 (+ 1354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:20.000515 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:20.000519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:20.000717 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:20.001492 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:20.001498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:20.001502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:20.001973 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:20.002158 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:20.005149 (+ 2991us) server_negotiation.cc:299] Negotiation successful
0812 00:37:20.005524 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1200,"thread_start_us":1064,"threads_started":1}
I20260812 00:37:20.006562 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:19.976093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:37481 (local address 127.7.244.66:43883)
0812 00:37:19.980139 (+ 4046us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:19.980210 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:19.982792 (+ 2582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:19.983883 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:19.983897 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:19.984582 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:19.985589 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:19.985621 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:19.987335 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:19.987341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:19.989494 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:19.989519 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:19.990892 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:19.990900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:19.991212 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:19.992136 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:19.992174 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:19.994908 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:19.999451 (+ 4543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:19.999464 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:19.999489 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:20.000257 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:20.000894 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:20.000900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:20.000903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:20.001256 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:20.002258 (+ 1002us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:20.002270 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:20.002894 (+ 624us) client_negotiation.cc:769] Sending connection context
0812 00:37:20.004042 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0812 00:37:20.004847 (+ 805us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3723,"thread_start_us":3450,"threads_started":1}
I20260812 00:37:20.008636 2932 heartbeater.cc:347] Connected to a master server at 127.7.244.126:37481
I20260812 00:37:20.009004 2932 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:20.009985 2932 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
I20260812 00:37:20.012532 2220 ts_manager.cc:194] Registered new tserver with Master: b8300a32716e4804a04fe82e5e0844f3 (127.7.244.66:35119)
I20260812 00:37:20.015165 2220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:43883
Aug 12 00:37:20 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:20.329422 2631 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:20.329927 2631 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
I20260812 00:37:20.331367 2220 ts_manager.cc:194] Registered new tserver with Master: 7dbba47ad58d4cfd8c70cf8de98f6712 (127.7.244.65:36221)
I20260812 00:37:20.332592 2220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:45039
W20260812 00:37:20.435900 2940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:20.436430 2940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:20.436671 2940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:20.448415 2940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:20.448588 2940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:20.448786 2940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:37:20.462682 2940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:39891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.244.84:46469/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:45251
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=35161
--tserver_master_addrs=127.7.244.126:37481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:20.464138 2940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:20.465719 2940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:20.485766 2953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:20.486462 2957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:20.496223 2940 server_base.cc:1034] running on GCE node
W20260812 00:37:20.496728 2963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:20.497547 2940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:20.500057 2940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:20.501369 2940 hybrid_clock.cc:648] HybridClock initialized: now 1786495040501322 us; error 49 us; skew 500 ppm
I20260812 00:37:20.501744 2940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:20 dist-test-slave-gjw7 krb5kdc[32546](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495040, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:20.506394 2940 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:37:20.516381 2940 webserver.cc:460] Webserver started at http://127.7.244.67:35161/ using document root <none> and password file <none>
I20260812 00:37:20.517225 2940 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:20.517423 2940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 00:37:20 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495040, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:20.559844 2940 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.008s sys 0.004s
I20260812 00:37:20.579197 2975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:20.582329 2940 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.006s sys 0.008s
I20260812 00:37:20.582726 2940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "08cd113a026d44769b722d413d70fc1e"
format_stamp: "Formatted at 2026-08-12 00:37:17 on dist-test-slave-gjw7"
server_key: "a06d04fec171ee2e1ef143a4035d53b6"
server_key_iv: "8a777df28f1a9a6f71222d6668d53327"
server_key_version: "kuduclusterkey@0"
I20260812 00:37:20.584698 2940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:20.635486 2940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:20.641163 2940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:20.641557 2940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:20.646294 2940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:20.648202 2940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:20.648320 2940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:20.648480 2940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:20.648640 2940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 00:37:20 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for HTTP/127.7.244.84@KRBTEST.COM
I20260812 00:37:20.839960 2940 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:45251
I20260812 00:37:20.840030 3182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:45251 every 8 connection(s)
I20260812 00:37:20.842115 2940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:37:20.850196 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 2940
I20260812 00:37:20.896477 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:20.851981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.7.244.126:37481)
0812 00:37:20.870731 (+ 18750us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:20.870739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:20.873159 (+ 2420us) server_negotiation.cc:407] Connection header received
0812 00:37:20.873280 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:20.873314 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:20.873423 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:20.873572 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:20.878602 (+ 5030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:20.879963 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:20.882252 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:20.883007 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:20.883132 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:20.887561 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:20.887571 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:20.887576 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:20.887641 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:20.891942 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:20.893174 (+ 1232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:20.893182 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:20.893185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:20.893432 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:20.894016 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:20.894022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:20.894026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:20.894608 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:20.894903 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:20.895697 (+ 794us) server_negotiation.cc:299] Negotiation successful
0812 00:37:20.896045 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18594,"thread_start_us":18446,"threads_started":1}
Aug 12 00:37:20 dist-test-slave-gjw7 krb5kdc[32546](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495040, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:20.918352 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:20.875855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:57887 (local address 127.7.244.126:37481)
0812 00:37:20.892579 (+ 16724us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:20.892586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:20.892609 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:37:20.892684 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:20.892689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:20.892775 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:20.892897 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:20.896248 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:20.897972 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:20.901899 (+ 3927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:20.902613 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:20.902774 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:20.907844 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:20.907855 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:20.907860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:20.907914 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:20.911704 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:20.913274 (+ 1570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:20.913280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:20.913284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:20.913524 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:20.914370 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:20.914375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:20.914378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:20.915035 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:20.915174 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:20.917695 (+ 2521us) server_negotiation.cc:299] Negotiation successful
0812 00:37:20.917961 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16572,"thread_start_us":17196,"threads_started":1}
I20260812 00:37:20.919412 3194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:20.876713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:37481 (local address 127.7.244.67:57887)
0812 00:37:20.885062 (+ 8349us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:20.885142 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:20.887793 (+ 2651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:20.893687 (+ 5894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:20.893703 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:20.894700 (+ 997us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:20.895900 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:20.895970 (+ 70us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:20.898216 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:20.898508 (+ 292us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:20.901671 (+ 3163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:20.901697 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:20.902947 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:20.902956 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:20.903312 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:20.904297 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:20.904332 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:20.907386 (+ 3054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:20.912259 (+ 4873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:20.912272 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:20.912296 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:20.913083 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:20.913684 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:20.913690 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:20.913694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:20.914220 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:20.915549 (+ 1329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:20.915558 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:20.916111 (+ 553us) client_negotiation.cc:769] Sending connection context
0812 00:37:20.916490 (+ 379us) client_negotiation.cc:240] Negotiation successful
0812 00:37:20.917194 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7924,"thread_start_us":12452,"threads_started":1}
I20260812 00:37:20.921202 3183 heartbeater.cc:347] Connected to a master server at 127.7.244.126:37481
I20260812 00:37:20.921612 3183 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:20.922571 3183 heartbeater.cc:511] Master 127.7.244.126:37481 requested a full tablet report, sending...
I20260812 00:37:20.924547 2220 ts_manager.cc:194] Registered new tserver with Master: 08cd113a026d44769b722d413d70fc1e (127.7.244.67:45251)
I20260812 00:37:20.925866 2220 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:57887
I20260812 00:37:20.931843 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:37:20.950670 3193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:20.935798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.7.244.126:37481)
0812 00:37:20.936046 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:20.936053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:20.936511 (+ 458us) server_negotiation.cc:407] Connection header received
0812 00:37:20.936663 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:20.936669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:20.936760 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:20.936882 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:20.939085 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:20.939821 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:20.941319 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:20.941770 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:20.941907 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:20.944739 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:20.944744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:20.944747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:20.944790 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:20.947658 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:20.948504 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:20.948509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:20.948511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:20.948660 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:20.949092 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:20.949096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:20.949098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:20.949470 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:20.949712 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:20.950095 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 00:37:20.950341 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 00:37:20.957607 2220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:37:20.969365 2220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:37:21.020561 2932 heartbeater.cc:503] Master 127.7.244.126:37481 was elected leader, sending a full tablet report...
I20260812 00:37:21.024514 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:20.988552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.7.244.66:35119)
0812 00:37:21.012305 (+ 23753us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.012316 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.013637 (+ 1321us) server_negotiation.cc:407] Connection header received
0812 00:37:21.013779 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.013790 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.014121 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.014336 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:21.015446 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.017013 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.020102 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.022327 (+ 2225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.022461 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.022733 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.023977 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.024173 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23516,"thread_start_us":23779,"threads_started":1}
I20260812 00:37:21.026723 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:20.987902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:35119 (local address 127.0.0.1:43916)
0812 00:37:21.013013 (+ 25111us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:21.013102 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:21.013384 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:21.014677 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:21.014686 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:21.014731 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:21.015186 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.015201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.017207 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.017213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.019661 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.019678 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.022915 (+ 3237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.022923 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.023156 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.023227 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 00:37:21.023373 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 00:37:21.023601 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24949,"thread_start_us":31906,"threads_started":1}
I20260812 00:37:21.030329 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.008721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:36221 (local address 127.0.0.1:36108)
0812 00:37:21.016436 (+ 7715us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:21.016457 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:21.016579 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:21.021010 (+ 4431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:21.021016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:21.021048 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:21.021570 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.021586 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.024625 (+ 3039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.024631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.027415 (+ 2784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.027434 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.029493 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.029500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.029680 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.029717 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 00:37:21.029797 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 00:37:21.029946 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7560,"thread_start_us":7918,"threads_started":1}
I20260812 00:37:21.031600 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.002810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36108 (local address 127.7.244.65:36221)
0812 00:37:21.019318 (+ 16508us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.019328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.019361 (+ 33us) server_negotiation.cc:407] Connection header received
0812 00:37:21.019486 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.019495 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.019822 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.020054 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:21.022888 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.024410 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.027639 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.029192 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.029363 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.030981 (+ 1618us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.031076 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.031268 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16301,"thread_start_us":19699,"threads_started":1}
I20260812 00:37:21.032028 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 2580cf02e5ea45499fd6a9204a0dc114 (DEFAULT_TABLE table=test-table [id=69ffb9b051db43e6839cd57127c90b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:21.033931 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2580cf02e5ea45499fd6a9204a0dc114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:21.036346 2501 tablet_service.cc:1467] Processing CreateTablet for tablet 2580cf02e5ea45499fd6a9204a0dc114 (DEFAULT_TABLE table=test-table [id=69ffb9b051db43e6839cd57127c90b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:21.038154 2501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2580cf02e5ea45499fd6a9204a0dc114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:21.051714 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:20.987166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:45251 (local address 127.0.0.1:40668)
0812 00:37:21.014139 (+ 26973us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:21.014164 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:21.014304 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:21.026186 (+ 11882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:21.026190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:21.026232 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:21.026615 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.026627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.044321 (+ 17694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.044329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.046586 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.046613 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.048815 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.048825 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.049058 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.051022 (+ 1964us) client_negotiation.cc:769] Sending connection context
0812 00:37:21.051105 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 00:37:21.051292 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13767,"thread_start_us":14138,"threads_started":1}
I20260812 00:37:21.053503 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:20.987537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.7.244.67:45251)
0812 00:37:21.025418 (+ 37881us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.025428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.025468 (+ 40us) server_negotiation.cc:407] Connection header received
0812 00:37:21.025581 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.025589 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.025921 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.026166 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:21.027703 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.044085 (+ 16382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.046913 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.048475 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.052421 (+ 3946us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.052748 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.052844 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.053052 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37636,"thread_start_us":37472,"threads_started":1}
I20260812 00:37:21.055246 3246 tablet_bootstrap.cc:492] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Bootstrap starting.
I20260812 00:37:21.058163 3060 tablet_service.cc:1467] Processing CreateTablet for tablet 2580cf02e5ea45499fd6a9204a0dc114 (DEFAULT_TABLE table=test-table [id=69ffb9b051db43e6839cd57127c90b4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:21.061550 3060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2580cf02e5ea45499fd6a9204a0dc114. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:21.061800 3246 tablet_bootstrap.cc:654] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:21.063294 3246 log.cc:824] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:21.065454 3247 tablet_bootstrap.cc:492] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Bootstrap starting.
I20260812 00:37:21.069237 3247 tablet_bootstrap.cc:654] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:21.077771 3247 log.cc:824] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:21.080163 3246 tablet_bootstrap.cc:492] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: No bootstrap required, opened a new log
I20260812 00:37:21.080514 3246 ts_tablet_manager.cc:1397] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Time spent bootstrapping tablet: real 0.026s user 0.015s sys 0.005s
I20260812 00:37:21.088090 3247 tablet_bootstrap.cc:492] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: No bootstrap required, opened a new log
I20260812 00:37:21.088572 3247 ts_tablet_manager.cc:1397] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.004s
I20260812 00:37:21.092759 3253 tablet_bootstrap.cc:492] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: Bootstrap starting.
I20260812 00:37:21.093838 3246 raft_consensus.cc:348] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.094728 3246 raft_consensus.cc:374] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:21.094887 3246 raft_consensus.cc:729] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8300a32716e4804a04fe82e5e0844f3, State: Initialized, Role: FOLLOWER
I20260812 00:37:21.095952 3246 consensus_queue.cc:260] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.097646 3253 tablet_bootstrap.cc:654] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:21.099062 3253 log.cc:824] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:21.102386 3247 raft_consensus.cc:348] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.103061 3247 raft_consensus.cc:374] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:21.103247 3247 raft_consensus.cc:729] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dbba47ad58d4cfd8c70cf8de98f6712, State: Initialized, Role: FOLLOWER
I20260812 00:37:21.104229 3247 consensus_queue.cc:260] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.106626 3246 ts_tablet_manager.cc:1428] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Time spent starting tablet: real 0.026s user 0.011s sys 0.012s
I20260812 00:37:21.109215 2631 heartbeater.cc:503] Master 127.7.244.126:37481 was elected leader, sending a full tablet report...
I20260812 00:37:21.110857 3247 ts_tablet_manager.cc:1428] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Time spent starting tablet: real 0.022s user 0.015s sys 0.000s
I20260812 00:37:21.114655 3253 tablet_bootstrap.cc:492] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: No bootstrap required, opened a new log
I20260812 00:37:21.115248 3253 ts_tablet_manager.cc:1397] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: Time spent bootstrapping tablet: real 0.023s user 0.013s sys 0.004s
I20260812 00:37:21.126096 3253 raft_consensus.cc:348] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.126633 3253 raft_consensus.cc:374] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:21.126735 3253 raft_consensus.cc:729] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08cd113a026d44769b722d413d70fc1e, State: Initialized, Role: FOLLOWER
I20260812 00:37:21.127455 3253 consensus_queue.cc:260] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.128661 3183 heartbeater.cc:503] Master 127.7.244.126:37481 was elected leader, sending a full tablet report...
I20260812 00:37:21.129354 3253 ts_tablet_manager.cc:1428] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260812 00:37:21.230710 2936 tablet.cc:2369] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:37:21.270699 3260 raft_consensus.cc:482] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:37:21.271059 3260 raft_consensus.cc:504] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.272902 3260 leader_election.cc:288] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08cd113a026d44769b722d413d70fc1e (127.7.244.67:45251), b8300a32716e4804a04fe82e5e0844f3 (127.7.244.66:35119)
W20260812 00:37:21.279647 2632 tablet.cc:2369] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:37:21.282732 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.273468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:49463 (local address 127.7.244.66:35119)
0812 00:37:21.273682 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.273688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.274995 (+ 1307us) server_negotiation.cc:407] Connection header received
0812 00:37:21.275128 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.275134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.275211 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.275330 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:21.276183 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.277022 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.279967 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.281196 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.281384 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.281467 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.282294 (+ 827us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.282460 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 00:37:21.282898 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.273439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:45251 (local address 127.7.244.65:59729)
0812 00:37:21.274591 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:21.274617 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:21.274879 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:21.275370 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:21.275375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:21.275490 (+ 115us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:21.275980 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.275990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.277633 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.277639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.279775 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.279785 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.281747 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.281752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.282016 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.282112 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 00:37:21.282219 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 00:37:21.282412 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":943,"mutex_wait_us":101,"spinlock_wait_cycles":29952,"thread_start_us":1105,"threads_started":1}
I20260812 00:37:21.282936 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.273696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:35119 (local address 127.7.244.65:49463)
0812 00:37:21.274861 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:21.274877 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:21.275002 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:21.275370 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:21.275375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:21.275405 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:21.275954 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.275974 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.277253 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.277259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.279719 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:21.279745 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.281728 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.281736 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:21.282040 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.282116 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 00:37:21.282219 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 00:37:21.282406 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"mutex_wait_us":177,"spinlock_wait_cycles":20352,"thread_start_us":495,"threads_started":1}
I20260812 00:37:21.284368 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.273969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:59729 (local address 127.7.244.67:45251)
0812 00:37:21.274169 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.274175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.274740 (+ 565us) server_negotiation.cc:407] Connection header received
0812 00:37:21.275061 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.275066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.275144 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.275194 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:21.276150 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.277485 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.279953 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.281196 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.281384 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.281467 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.283950 (+ 2483us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.284118 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 00:37:21.284235 2835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2580cf02e5ea45499fd6a9204a0dc114" candidate_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8300a32716e4804a04fe82e5e0844f3" is_pre_election: true
I20260812 00:37:21.284847 2835 raft_consensus.cc:2454] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dbba47ad58d4cfd8c70cf8de98f6712 in term 0.
I20260812 00:37:21.285393 3095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2580cf02e5ea45499fd6a9204a0dc114" candidate_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08cd113a026d44769b722d413d70fc1e" is_pre_election: true
I20260812 00:37:21.285779 2395 leader_election.cc:302] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dbba47ad58d4cfd8c70cf8de98f6712, b8300a32716e4804a04fe82e5e0844f3; no voters:
I20260812 00:37:21.286060 3095 raft_consensus.cc:2454] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dbba47ad58d4cfd8c70cf8de98f6712 in term 0.
I20260812 00:37:21.286293 3260 raft_consensus.cc:2781] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:37:21.286450 3260 raft_consensus.cc:482] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:37:21.286548 3260 raft_consensus.cc:3037] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:21.289731 3260 raft_consensus.cc:504] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.290573 3260 leader_election.cc:288] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [CANDIDATE]: Term 1 election: Requested vote from peers 08cd113a026d44769b722d413d70fc1e (127.7.244.67:45251), b8300a32716e4804a04fe82e5e0844f3 (127.7.244.66:35119)
I20260812 00:37:21.291134 2835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2580cf02e5ea45499fd6a9204a0dc114" candidate_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8300a32716e4804a04fe82e5e0844f3"
I20260812 00:37:21.291165 3095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2580cf02e5ea45499fd6a9204a0dc114" candidate_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08cd113a026d44769b722d413d70fc1e"
I20260812 00:37:21.291370 2835 raft_consensus.cc:3037] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:21.291394 3095 raft_consensus.cc:3037] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:21.294271 2835 raft_consensus.cc:2454] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dbba47ad58d4cfd8c70cf8de98f6712 in term 1.
I20260812 00:37:21.294945 2395 leader_election.cc:302] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dbba47ad58d4cfd8c70cf8de98f6712, b8300a32716e4804a04fe82e5e0844f3; no voters:
I20260812 00:37:21.295310 3260 raft_consensus.cc:2781] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:37:21.295902 3260 raft_consensus.cc:686] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 1 LEADER]: Becoming Leader. State: Replica: 7dbba47ad58d4cfd8c70cf8de98f6712, State: Running, Role: LEADER
I20260812 00:37:21.296600 3260 consensus_queue.cc:237] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } }
I20260812 00:37:21.298190 3095 raft_consensus.cc:2454] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dbba47ad58d4cfd8c70cf8de98f6712 in term 1.
I20260812 00:37:21.301810 2220 catalog_manager.cc:5495] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dbba47ad58d4cfd8c70cf8de98f6712 (127.7.244.65). New cstate: current_term: 1 leader_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dbba47ad58d4cfd8c70cf8de98f6712" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 36221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 } health_report { overall_health: UNKNOWN } } }
I20260812 00:37:21.333702 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.327823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36118 (local address 127.7.244.65:36221)
0812 00:37:21.328058 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.328064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.328110 (+ 46us) server_negotiation.cc:407] Connection header received
0812 00:37:21.328303 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.328308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.328400 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.328544 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:37:21.328959 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.329751 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.331225 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.331687 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.331804 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.332411 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:37:21.333014 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:37:21.333160 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.333330 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.333426 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:37:21.342532 2835 raft_consensus.cc:1261] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 1 FOLLOWER]: Refusing update from remote peer 7dbba47ad58d4cfd8c70cf8de98f6712: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:37:21.342536 3095 raft_consensus.cc:1261] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 1 FOLLOWER]: Refusing update from remote peer 7dbba47ad58d4cfd8c70cf8de98f6712: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:37:21.343590 3260 consensus_queue.cc:1035] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8300a32716e4804a04fe82e5e0844f3" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:37:21.343910 3270 consensus_queue.cc:1035] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08cd113a026d44769b722d413d70fc1e" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 45251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:37:21.375509 2216 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:37:21.375969 2216 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:37:21.381403 2216 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 850c6aa02cb54b04be606cda5cdaba26: Sending DeleteTablet for 3 replicas of tablet 2580cf02e5ea45499fd6a9204a0dc114
I20260812 00:37:21.383005 2501 tablet_service.cc:1514] Processing DeleteTablet for tablet 2580cf02e5ea45499fd6a9204a0dc114 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:21 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:36108
I20260812 00:37:21.382951 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet 2580cf02e5ea45499fd6a9204a0dc114 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:21 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:43916
I20260812 00:37:21.383003 3060 tablet_service.cc:1514] Processing DeleteTablet for tablet 2580cf02e5ea45499fd6a9204a0dc114 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:21 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:40668
I20260812 00:37:21.396018 3299 tablet_replica.cc:330] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: stopping tablet replica
I20260812 00:37:21.396340 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 2293
I20260812 00:37:21.396898 3299 raft_consensus.cc:2229] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:37:21.397763 3299 raft_consensus.cc:2258] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:37:21.398831 3303 tablet_replica.cc:330] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: stopping tablet replica
I20260812 00:37:21.399358 3303 raft_consensus.cc:2229] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:37:21.399963 3303 raft_consensus.cc:2258] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:37:21.402719 3303 ts_tablet_manager.cc:1905] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:37:21.404714 3299 ts_tablet_manager.cc:1905] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:37:21.410976 3303 ts_tablet_manager.cc:1918] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:37:21.411500 3303 log.cc:1196] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/2580cf02e5ea45499fd6a9204a0dc114
I20260812 00:37:21.412767 3303 ts_tablet_manager.cc:1939] T 2580cf02e5ea45499fd6a9204a0dc114 P 08cd113a026d44769b722d413d70fc1e: Deleting consensus metadata
I20260812 00:37:21.415263 2195 catalog_manager.cc:4841] TS 08cd113a026d44769b722d413d70fc1e (127.7.244.67:45251): tablet 2580cf02e5ea45499fd6a9204a0dc114 (table test-table [id=69ffb9b051db43e6839cd57127c90b4c]) successfully deleted
I20260812 00:37:21.416029 3298 tablet_replica.cc:330] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: stopping tablet replica
I20260812 00:37:21.416458 3298 raft_consensus.cc:2229] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:37:21.416949 3298 raft_consensus.cc:2258] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:37:21.419222 3298 ts_tablet_manager.cc:1905] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:37:21.421442 3299 ts_tablet_manager.cc:1918] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:37:21.421703 3299 log.cc:1196] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/2580cf02e5ea45499fd6a9204a0dc114
I20260812 00:37:21.422634 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.399906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36122 (local address 127.7.244.65:36221)
0812 00:37:21.400111 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.400118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.400627 (+ 509us) server_negotiation.cc:407] Connection header received
0812 00:37:21.401527 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.401534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.401638 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.401726 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:21.403900 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.405141 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.407267 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.408023 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.408136 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.412141 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:21.412162 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:21.412184 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:21.412242 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:21.416736 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:21.418543 (+ 1807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:21.418554 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:21.418557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:21.418802 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:21.419361 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:21.419367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:21.419374 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:21.420225 (+ 851us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:21.420552 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.421231 (+ 679us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.421652 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:37:21.423154 3299 ts_tablet_manager.cc:1939] T 2580cf02e5ea45499fd6a9204a0dc114 P 7dbba47ad58d4cfd8c70cf8de98f6712: Deleting consensus metadata
I20260812 00:37:21.423581 2613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:21.426539 3298 ts_tablet_manager.cc:1918] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:37:21.426815 3298 log.cc:1196] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/2580cf02e5ea45499fd6a9204a0dc114
I20260812 00:37:21.427438 3298 ts_tablet_manager.cc:1939] T 2580cf02e5ea45499fd6a9204a0dc114 P b8300a32716e4804a04fe82e5e0844f3: Deleting consensus metadata
I20260812 00:37:21.429695 2197 catalog_manager.cc:4841] TS b8300a32716e4804a04fe82e5e0844f3 (127.7.244.66:35119): tablet 2580cf02e5ea45499fd6a9204a0dc114 (table test-table [id=69ffb9b051db43e6839cd57127c90b4c]) successfully deleted
I20260812 00:37:21.535944 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 2293
I20260812 00:37:21.536069 2195 catalog_manager.cc:4841] TS 7dbba47ad58d4cfd8c70cf8de98f6712 (127.7.244.65:36221): tablet 2580cf02e5ea45499fd6a9204a0dc114 (table test-table [id=69ffb9b051db43e6839cd57127c90b4c]) successfully deleted
I20260812 00:37:21.559942 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 2642
I20260812 00:37:21.574747 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.560842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.7.244.66:35119)
0812 00:37:21.561020 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.561024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.561045 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:37:21.561171 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.561176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.561242 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.561366 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:21.562770 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.563542 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.564894 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.565518 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.565644 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.568567 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:21.568586 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:21.568606 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:21.568657 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:21.571702 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:21.572537 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:21.572545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:21.572548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:21.572706 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:21.573204 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:21.573208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:21.573210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:21.573670 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:21.573930 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.574209 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.574406 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:37:21.575541 2923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:21.665923 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 2642
I20260812 00:37:21.694387 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 2940
I20260812 00:37:21.709215 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:21.695336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.7.244.67:45251)
0812 00:37:21.695531 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:21.695535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:21.695728 (+ 193us) server_negotiation.cc:407] Connection header received
0812 00:37:21.695880 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:21.695884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:21.695992 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:21.696081 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:21.697658 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.698391 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.699781 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:21.700179 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:21.700281 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:21.703133 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:21.703149 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:21.703168 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:21.703204 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:21.706173 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:21.706994 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:21.707001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:21.707004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:21.707147 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:21.707648 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:21.707652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:21.707654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:21.708046 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:21.708321 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:21.708597 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 00:37:21.708780 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 00:37:21.710045 3176 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:21.800268 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 2940
I20260812 00:37:21.830973 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 2154
I20260812 00:37:21.832360 2278 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:21.951138 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 2154
I20260812 00:37:21.993813 8145 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 00:37:22.330397 8145 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 00:37:22.330611 8145 mini_ranger.cc:66] Stopping Ranger...
I20260812 00:37:22.690699 8145 mini_ranger.cc:68] Stopped Ranger
2026-08-12 00:37:22.691 UTC [32610] LOG: received smart shutdown request
2026-08-12 00:37:22.694 UTC [32610] LOG: background worker "logical replication launcher" (PID 32628) exited with exit code 1
2026-08-12 00:37:22.694 UTC [32619] LOG: shutting down
2026-08-12 00:37:22.895 UTC [32610] LOG: database system is shut down
2026-08-12T00:37:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56199 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:37:23 dist-test-slave-gjw7 krb5kdc[3333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:37:23 dist-test-slave-gjw7 krb5kdc[3333](info): set up 2 sockets
Aug 12 00:37:23 dist-test-slave-gjw7 krb5kdc[3333](info): commencing operation
krb5kdc: starting...
W20260812 00:37:24.338706 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:37:24 dist-test-slave-gjw7 krb5kdc[3333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:37:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:37:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:37:24.626057 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42533
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:43969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42533,127.7.244.125:44903,127.7.244.124:39273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:24.997731 3354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:24.998088 3354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:24.998184 3354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:25.006625 3354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:25.006786 3354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:25.006837 3354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:25.006898 3354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:25.006943 3354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:25.017977 3354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:43969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42533,127.7.244.125:44903,127.7.244.124:39273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:25.019455 3354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:25.020941 3354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:25.033957 3367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:25.033923 3372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:25.034009 3368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:25.035145 3354 server_base.cc:1034] running on GCE node
I20260812 00:37:25.036134 3354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:25.037897 3354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:25.039177 3354 hybrid_clock.cc:648] HybridClock initialized: now 1786495045039119 us; error 60 us; skew 500 ppm
I20260812 00:37:25.039594 3354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:25 dist-test-slave-gjw7 krb5kdc[3333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:25.047185 3354 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:37:25.048957 3354 webserver.cc:460] Webserver started at http://127.7.244.126:36807/ using document root <none> and password file <none>
I20260812 00:37:25.049820 3354 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:25.049973 3354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:25.050299 3354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:25.053650 3354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "937d9027ec00447eb8be755956a6d4f4"
format_stamp: "Formatted at 2026-08-12 00:37:25 on dist-test-slave-gjw7"
server_key: "317ee08d563976b94606a413127e8fe3"
server_key_iv: "3029c7ee348a3474d86ef674b18087fb"
server_key_version: "encryptionkey@0"
I20260812 00:37:25.054632 3354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "937d9027ec00447eb8be755956a6d4f4"
format_stamp: "Formatted at 2026-08-12 00:37:25 on dist-test-slave-gjw7"
server_key: "317ee08d563976b94606a413127e8fe3"
server_key_iv: "3029c7ee348a3474d86ef674b18087fb"
server_key_version: "encryptionkey@0"
I20260812 00:37:25.061158 3354 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:37:25.065877 3390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:25.067706 3354 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260812 00:37:25.067976 3354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "937d9027ec00447eb8be755956a6d4f4"
format_stamp: "Formatted at 2026-08-12 00:37:25 on dist-test-slave-gjw7"
server_key: "317ee08d563976b94606a413127e8fe3"
server_key_iv: "3029c7ee348a3474d86ef674b18087fb"
server_key_version: "encryptionkey@0"
I20260812 00:37:25.068222 3354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:25.090647 3354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:25.095393 3354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:25.095721 3354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:25.114812 3354 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:42533
I20260812 00:37:25.114847 3499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:42533 every 8 connection(s)
I20260812 00:37:25.116575 3354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:37:25.120872 3501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:25.126657 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 3354
I20260812 00:37:25.126870 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:37:25.127521 8145 external_mini_cluster.cc:1428] Setting key 1b54caa77c135c936c2c8e393854a5c9
I20260812 00:37:25.128429 3501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:37:25 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.244.125@KRBTEST.COM; defaulting to no policy
I20260812 00:37:25.156514 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.132220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42533 (local address 127.0.0.1:41584)
0812 00:37:25.133580 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.133625 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:25.135560 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:25.136607 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:25.136614 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:25.137544 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:25.138087 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:25.138121 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.140109 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.140116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:25.141926 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:25.141948 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.143120 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.143128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:25.143307 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:25.144136 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:25.144168 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:25.147231 (+ 3063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:25.151641 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:25.151655 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:25.151666 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:25.152146 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:25.152791 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:25.152795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:25.152797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:25.152981 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:25.154188 (+ 1207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:25.154195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:25.154433 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 00:37:25.154709 (+ 276us) client_negotiation.cc:240] Negotiation successful
0812 00:37:25.155210 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"thread_start_us":844,"threads_started":1}
I20260812 00:37:25.156565 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.132216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.7.244.126:42533)
0812 00:37:25.133220 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:25.133241 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:25.133783 (+ 542us) server_negotiation.cc:407] Connection header received
0812 00:37:25.136046 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:25.136053 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:25.136332 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:25.136537 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:25.138417 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.139863 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.142146 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.142817 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.142987 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:25.147600 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:25.147611 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:25.147626 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:25.147670 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:25.151414 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:25.152367 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:25.152377 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:25.152380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:25.152639 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:25.153133 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:25.153139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:25.153142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:25.154006 (+ 864us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:25.154254 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:25.154933 (+ 679us) server_negotiation.cc:299] Negotiation successful
0812 00:37:25.155442 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":217,"server-negotiator.queue_time_us":713,"thread_start_us":447,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.125@KRBTEST.COM" created.
I20260812 00:37:25.160256 3501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:37:25.161619 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.160996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39692)
0812 00:37:25.161278 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.161462 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 00:37:25.161948 3392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
I20260812 00:37:25.163882 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.163352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39698)
0812 00:37:25.163558 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.163676 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260812 00:37:25.164119 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:44903: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab.
Entry for principal kudu/127.7.244.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab.
I20260812 00:37:25.194370 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.125:44903
--webserver_interface=127.7.244.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab
--principal=kudu/127.7.244.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:43969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42533,127.7.244.125:44903,127.7.244.124:39273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:37:25.213512 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } attempt: 1
I20260812 00:37:25.214789 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.214310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39704)
0812 00:37:25.214530 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.214645 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 00:37:25.215937 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.215532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39710)
0812 00:37:25.215698 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.215773 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 00:37:25.216153 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:44903: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
I20260812 00:37:25.284469 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } attempt: 2
I20260812 00:37:25.285935 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.285195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39716)
0812 00:37:25.285568 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.285724 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
I20260812 00:37:25.287321 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.286799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39722)
0812 00:37:25.286989 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.287118 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 00:37:25.287595 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:44903: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
I20260812 00:37:25.378957 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } attempt: 3
I20260812 00:37:25.380342 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.379750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39736)
0812 00:37:25.380017 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.380152 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260812 00:37:25.381628 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.381095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39744)
0812 00:37:25.381268 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.381433 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 00:37:25.381932 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:44903: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
I20260812 00:37:25.525267 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } attempt: 4
I20260812 00:37:25.526794 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.526070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39754)
0812 00:37:25.526434 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.526578 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260812 00:37:25.528002 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.527541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39758)
0812 00:37:25.527708 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.527836 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 00:37:25.528249 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:44903: Network error: Client connection negotiation failed: client connection to 127.7.244.125:44903: connect: Connection refused (error 111)
W20260812 00:37:25.558966 3512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:25.559315 3512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:25.559410 3512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:25.567250 3512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:25.567390 3512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:25.567435 3512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:25.567497 3512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:25.567540 3512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:25.578286 3512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:43969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42533,127.7.244.125:44903,127.7.244.124:39273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab
--principal=kudu/127.7.244.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.125:44903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.244.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:25.579747 3512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:25.581104 3512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:25.594424 3521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:25.594533 3526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:25.594427 3520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:25.595383 3512 server_base.cc:1034] running on GCE node
I20260812 00:37:25.596357 3512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:25.598085 3512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:25.599375 3512 hybrid_clock.cc:648] HybridClock initialized: now 1786495045599306 us; error 59 us; skew 500 ppm
I20260812 00:37:25.599777 3512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:25 dist-test-slave-gjw7 krb5kdc[3333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:25.603346 3512 init.cc:377] Logged in from keytab as kudu/127.7.244.125@KRBTEST.COM (short username kudu)
I20260812 00:37:25.605170 3512 webserver.cc:460] Webserver started at http://127.7.244.125:36863/ using document root <none> and password file <none>
I20260812 00:37:25.605968 3512 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:25.606106 3512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:25.606434 3512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:25.609694 3512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/data/instance:
uuid: "730ca0bf160347d085c7fb042d71e3fa"
format_stamp: "Formatted at 2026-08-12 00:37:25 on dist-test-slave-gjw7"
server_key: "d43f27927e05a06f132cfde853373fbf"
server_key_iv: "003b4f36a8532f4b70e6eaca004c1b9b"
server_key_version: "encryptionkey@0"
I20260812 00:37:25.610620 3512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/wal/instance:
uuid: "730ca0bf160347d085c7fb042d71e3fa"
format_stamp: "Formatted at 2026-08-12 00:37:25 on dist-test-slave-gjw7"
server_key: "d43f27927e05a06f132cfde853373fbf"
server_key_iv: "003b4f36a8532f4b70e6eaca004c1b9b"
server_key_version: "encryptionkey@0"
I20260812 00:37:25.616978 3512 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:37:25.620954 3554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:25.622531 3512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:37:25.622727 3512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/wal
uuid: "730ca0bf160347d085c7fb042d71e3fa"
format_stamp: "Formatted at 2026-08-12 00:37:25 on dist-test-slave-gjw7"
server_key: "d43f27927e05a06f132cfde853373fbf"
server_key_iv: "003b4f36a8532f4b70e6eaca004c1b9b"
server_key_version: "encryptionkey@0"
I20260812 00:37:25.622946 3512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:25.640743 3512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:25.645054 3512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:25.645368 3512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:25.664409 3512 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.125:44903
I20260812 00:37:25.664445 3637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.125:44903 every 8 connection(s)
I20260812 00:37:25.665912 3512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
I20260812 00:37:25.670104 3639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:25.673574 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 3512
I20260812 00:37:25.673772 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-1/wal/instance
I20260812 00:37:25.674397 8145 external_mini_cluster.cc:1428] Setting key fe150db8542f8a453906d7c2791d1595
I20260812 00:37:25.679127 3639 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:37:25 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.244.124@KRBTEST.COM; defaulting to no policy
I20260812 00:37:25.703904 3644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.682787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.7.244.126:42533)
0812 00:37:25.683350 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:25.683356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:25.684540 (+ 1184us) server_negotiation.cc:407] Connection header received
0812 00:37:25.686708 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:25.686713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:25.686798 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:25.686909 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:25.689228 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.690091 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.692608 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.693064 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.693188 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:25.697579 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:25.697585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:25.697588 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:25.697638 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:25.700456 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:25.701347 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:25.701352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:25.701354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:25.701487 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:25.701953 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:25.701957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:25.701959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:25.702291 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:25.702438 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:25.703259 (+ 821us) server_negotiation.cc:299] Negotiation successful
0812 00:37:25.703521 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":291,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.124@KRBTEST.COM" created.
I20260812 00:37:25.705071 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.683412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42533 (local address 127.0.0.1:41600)
0812 00:37:25.684318 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.684370 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:25.686357 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:25.687231 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:25.687244 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:25.688040 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:25.688916 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:25.688950 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.690325 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.690332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:25.692358 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:25.692391 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.693406 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.693414 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:25.693713 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:25.694524 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:25.694556 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:25.697222 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:25.700649 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:25.700659 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:25.700675 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:25.701177 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:25.701618 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:25.701622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:25.701624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:25.701821 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:25.702431 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:25.702440 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:25.702847 (+ 407us) client_negotiation.cc:769] Sending connection context
0812 00:37:25.703144 (+ 297us) client_negotiation.cc:240] Negotiation successful
0812 00:37:25.703740 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":382,"threads_started":1}
I20260812 00:37:25.707273 3639 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:37:25 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:37:25.723677 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.708143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39760)
0812 00:37:25.708313 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.708331 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:25.708435 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:25.709645 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:25.709649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:25.710116 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:25.710261 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:25.710270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.711667 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.711671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:25.712525 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:25.712538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.713501 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.713508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:25.713741 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:25.714352 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:25.714373 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:25.716802 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:25.720928 (+ 4126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:25.720933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:25.720936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:25.721453 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:25.721961 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:25.721965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:25.721968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:25.722134 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:25.722902 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:25.722905 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:25.723031 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 00:37:25.723146 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 00:37:25.723399 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 00:37:25.724031 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.708439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.7.244.125:44903)
0812 00:37:25.709113 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:25.709119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:25.709145 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:37:25.709197 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:25.709203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:25.709483 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:25.709629 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:25.710433 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.711513 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.712682 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.713274 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.713475 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:25.717072 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:25.717093 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:25.717115 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:25.717155 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:25.720743 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:25.721603 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:25.721615 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:25.721618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:25.721783 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:25.722271 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:25.722276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:25.722279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:25.722759 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:25.722968 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:25.723358 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 00:37:25.723636 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":389,"threads_started":1}
I20260812 00:37:25.726593 3639 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:37:25.727834 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.727306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56268)
0812 00:37:25.727488 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.727637 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 00:37:25.728068 3557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
I20260812 00:37:25.729761 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.729401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56276)
0812 00:37:25.729554 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.729648 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 00:37:25.729980 3639 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab.
Entry for principal kudu/127.7.244.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab.
I20260812 00:37:25.740899 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.124:39273
--webserver_interface=127.7.244.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab
--principal=kudu/127.7.244.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:43969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42533,127.7.244.125:44903,127.7.244.124:39273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:37:25.779325 3639 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 1
I20260812 00:37:25.780671 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.780144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56284)
0812 00:37:25.780362 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.780505 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 00:37:25.781782 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.781416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56296)
0812 00:37:25.781582 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.781660 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 00:37:25.782011 3639 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
I20260812 00:37:25.827546 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } attempt: 5
Aug 12 00:37:25 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:37:25.841841 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.828524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.7.244.125:44903)
0812 00:37:25.828696 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:25.828700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:25.828720 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:37:25.828861 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:25.828865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:25.828961 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:25.829083 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:25.830014 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.830711 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.831997 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.832454 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.832574 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:25.836157 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:25.836162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:25.836166 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:25.836203 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:25.838985 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:25.839929 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:25.839934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:25.839936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:25.840070 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:25.840493 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:25.840496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:25.840498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:25.840785 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:25.840917 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:25.841413 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 00:37:25.841606 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:37:25.842046 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.828323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39764)
0812 00:37:25.828559 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.828581 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:25.828736 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:25.829144 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:25.829150 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:25.829652 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:25.829839 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:25.829850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.830908 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.830914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:25.831825 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:25.831839 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:25.832624 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:25.832630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:25.832838 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:25.833508 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:25.833528 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:25.835866 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:25.839196 (+ 3330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:25.839202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:25.839207 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:25.839769 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:25.840194 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:25.840199 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:25.840202 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:25.840387 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:25.840918 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:25.840923 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:25.841114 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 00:37:25.841282 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 00:37:25.841705 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 00:37:25.843087 3501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:37:25.844204 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.843816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56310)
0812 00:37:25.843975 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.844062 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 00:37:25.845146 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.844838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56318)
0812 00:37:25.844960 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.845035 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 00:37:25.845398 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
I20260812 00:37:25.850214 3639 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 2
I20260812 00:37:25.851310 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.850829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56322)
0812 00:37:25.851031 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.851117 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 00:37:25.852321 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.851970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56330)
0812 00:37:25.852127 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.852200 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 00:37:25.852538 3639 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
I20260812 00:37:25.896651 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 1
I20260812 00:37:25.897953 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.897480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56332)
0812 00:37:25.897675 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.897773 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 00:37:25.899044 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.898690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56342)
0812 00:37:25.898851 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.898930 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 00:37:25.899281 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
I20260812 00:37:25.943750 3639 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 3
I20260812 00:37:25.944993 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.944471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56358)
0812 00:37:25.944716 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.944818 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 00:37:25.946424 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.945897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56360)
0812 00:37:25.946181 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.946277 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
W20260812 00:37:25.946695 3639 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
I20260812 00:37:25.967607 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 2
I20260812 00:37:25.968946 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.968406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56368)
0812 00:37:25.968660 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.968768 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 00:37:25.970160 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:25.969746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56378)
0812 00:37:25.969933 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:25.970031 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260812 00:37:25.970459 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
I20260812 00:37:26.076700 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 3
I20260812 00:37:26.077888 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.077411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56384)
0812 00:37:26.077613 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.077715 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 00:37:26.079200 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.078766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56390)
0812 00:37:26.078989 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.079076 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260812 00:37:26.079469 3501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
I20260812 00:37:26.089926 3639 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 4
I20260812 00:37:26.091360 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.090841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56404)
0812 00:37:26.091075 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.091180 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 00:37:26.092445 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.092073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56420)
0812 00:37:26.092233 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.092317 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 00:37:26.092677 3639 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:39273: Network error: Client connection negotiation failed: client connection to 127.7.244.124:39273: connect: Connection refused (error 111)
W20260812 00:37:26.104583 3655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:26.104939 3655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:26.105077 3655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:26.113126 3655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:26.113327 3655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:26.113430 3655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:26.113507 3655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:26.113587 3655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:26.128463 3655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:43969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42533,127.7.244.125:44903,127.7.244.124:39273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab
--principal=kudu/127.7.244.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.124:39273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.244.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:26.130081 3655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:26.131592 3655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:26.144578 3665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:26.144659 3667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:26.144603 3679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:26.145660 3655 server_base.cc:1034] running on GCE node
I20260812 00:37:26.146543 3655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:26.148209 3655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:26.149415 3655 hybrid_clock.cc:648] HybridClock initialized: now 1786495046149356 us; error 63 us; skew 500 ppm
I20260812 00:37:26.149816 3655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495046, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:26.153347 3655 init.cc:377] Logged in from keytab as kudu/127.7.244.124@KRBTEST.COM (short username kudu)
I20260812 00:37:26.155184 3655 webserver.cc:460] Webserver started at http://127.7.244.124:40571/ using document root <none> and password file <none>
I20260812 00:37:26.156006 3655 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:26.156165 3655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:26.156505 3655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:26.159968 3655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/data/instance:
uuid: "e451067b33ac4e1cbeeef618adce627a"
format_stamp: "Formatted at 2026-08-12 00:37:26 on dist-test-slave-gjw7"
server_key: "0c428261fdb74865e262f2f36e34f587"
server_key_iv: "3391fc20f5a9b4d8849cadfada3f1afd"
server_key_version: "encryptionkey@0"
I20260812 00:37:26.160934 3655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/wal/instance:
uuid: "e451067b33ac4e1cbeeef618adce627a"
format_stamp: "Formatted at 2026-08-12 00:37:26 on dist-test-slave-gjw7"
server_key: "0c428261fdb74865e262f2f36e34f587"
server_key_iv: "3391fc20f5a9b4d8849cadfada3f1afd"
server_key_version: "encryptionkey@0"
I20260812 00:37:26.166926 3655 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 00:37:26.170922 3691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:26.172588 3655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:37:26.172799 3655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/wal
uuid: "e451067b33ac4e1cbeeef618adce627a"
format_stamp: "Formatted at 2026-08-12 00:37:26 on dist-test-slave-gjw7"
server_key: "0c428261fdb74865e262f2f36e34f587"
server_key_iv: "3391fc20f5a9b4d8849cadfada3f1afd"
server_key_version: "encryptionkey@0"
I20260812 00:37:26.173024 3655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:26.191231 3655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:26.196022 3655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:26.196328 3655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:26.215253 3655 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.124:39273
I20260812 00:37:26.215291 3778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.124:39273 every 8 connection(s)
I20260812 00:37:26.216821 3655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
I20260812 00:37:26.219609 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 3655
I20260812 00:37:26.219827 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786494824837501-8145-0/minicluster-data/master-2/wal/instance
I20260812 00:37:26.220453 8145 external_mini_cluster.cc:1428] Setting key 2668a84bd79d624fc848d8d9441edfad
I20260812 00:37:26.221657 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:26.231568 3779 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495046, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:26.246899 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.222665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56430 (local address 127.7.244.124:39273)
0812 00:37:26.223571 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.223585 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.223637 (+ 52us) server_negotiation.cc:407] Connection header received
0812 00:37:26.225344 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.225391 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.225891 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.226291 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.228486 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.229975 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.231468 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.232257 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.232476 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.238116 (+ 5640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.238127 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.238144 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.238194 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.241979 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.243046 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.243054 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.243062 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.243247 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.243709 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.243713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.243715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.244318 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.244630 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.245128 (+ 498us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.245653 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":404,"threads_started":1}
I20260812 00:37:26.251405 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.234465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42533 (local address 127.0.0.1:41616)
0812 00:37:26.235274 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.235333 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:26.235518 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:26.236423 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:26.236430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:26.237281 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:26.237748 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.237762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.239258 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.239262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.240635 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.240653 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.241686 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.241691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.241837 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.242466 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:26.242490 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:26.244893 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:26.248264 (+ 3371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.248270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.248275 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.248771 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.249240 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.249245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.249248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.249467 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.250148 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:26.250156 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:26.250376 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 00:37:26.250593 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 00:37:26.250952 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":390,"threads_started":1}
I20260812 00:37:26.251415 3794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.234533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.7.244.126:42533)
0812 00:37:26.235977 (+ 1444us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.235983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.236003 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:37:26.236068 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.236073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.236155 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.236272 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.237978 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.239062 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.240826 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.241479 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.241625 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.245118 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.245123 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.245127 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.245163 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.248102 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.248914 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.248920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.248924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.249110 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.249585 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.249591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.249594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.250003 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.250149 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.250743 (+ 594us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.250972 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1295,"thread_start_us":1145,"threads_started":1}
I20260812 00:37:26.253233 3779 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:37:26.256665 3501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 4
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495046, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
I20260812 00:37:26.268870 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.253978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:44903 (local address 127.0.0.1:39770)
0812 00:37:26.254154 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.254173 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:26.254272 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:26.254666 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:26.254671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:26.255060 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:26.255210 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.255218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.256228 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.256233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.257467 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.257483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.258497 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.258503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.258707 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.259385 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:26.259415 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:26.262216 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:26.265936 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.265940 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.265943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.266364 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.266967 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.266973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.266976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.267138 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.267866 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:26.267871 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:26.268047 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 00:37:26.268195 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 00:37:26.268504 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 00:37:26.268916 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.254182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39770 (local address 127.7.244.125:44903)
0812 00:37:26.254395 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.254401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.254419 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:37:26.254468 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.254472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.254533 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.254616 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.255372 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.256077 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.257635 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.258121 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.258265 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.262442 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.262448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.262451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.262488 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.265758 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.266611 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.266617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.266621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.266804 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.267270 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.267275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.267277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.267640 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.267801 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.268336 (+ 535us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.268555 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:37:26.268934 3794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.250415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.7.244.126:42533)
0812 00:37:26.251674 (+ 1259us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.251680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.251697 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:37:26.251759 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.251765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.251867 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.251969 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.253722 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.254554 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.255899 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.256339 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.256461 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.262118 (+ 5657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.262124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.262129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.262176 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.265400 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.266351 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.266357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.266360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.266546 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.267045 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.267051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.267055 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.267445 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.267714 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.268374 (+ 660us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.268626 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":1764,"threads_started":1}
I20260812 00:37:26.270035 3779 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:37:26.275830 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.257275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56436)
0812 00:37:26.257725 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.257746 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:26.257857 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:26.258225 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:26.258242 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:26.258773 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:26.258989 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.259003 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.260376 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.260382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.261956 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.261974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.263338 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.263344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.263557 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.264241 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:26.264264 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:26.267162 (+ 2898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:26.272670 (+ 5508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.272677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.272681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.273265 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.273819 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.273824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.273827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.274006 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.274782 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:26.274786 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:26.274978 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 00:37:26.275125 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 00:37:26.275445 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337}
I20260812 00:37:26.275882 3785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.257539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56436 (local address 127.7.244.124:39273)
0812 00:37:26.257716 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.257721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.257814 (+ 93us) server_negotiation.cc:407] Connection header received
0812 00:37:26.257979 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.257984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.258064 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.258159 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.259143 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.260199 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.262097 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.262732 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.262864 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.267401 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.267408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.267413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.267449 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.272470 (+ 5021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.273462 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.273471 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.273475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.273677 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.274140 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.274145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.274148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.274571 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.274717 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.275265 (+ 548us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.275526 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495046, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:37:26.282480 3501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4: Bootstrap starting.
I20260812 00:37:26.287081 3786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.271724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56452)
0812 00:37:26.271900 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.271919 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:26.272025 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:26.273041 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:26.273046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:26.273525 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:26.273711 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.273724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.274867 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.274873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.276154 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.276170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.277161 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.277168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.277394 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.278068 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:26.278091 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:26.280881 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:26.284457 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.284462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.284466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.284935 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.285350 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.285355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.285359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.285504 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.286032 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:26.286037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:26.286184 (+ 147us) client_negotiation.cc:769] Sending connection context
0812 00:37:26.286307 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 00:37:26.286682 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 00:37:26.287108 3501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:26.287220 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.271916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56452 (local address 127.7.244.124:39273)
0812 00:37:26.272785 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.272791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.272811 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:37:26.272852 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.272855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.272905 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.272992 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.273874 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.274730 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.276310 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.276974 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.277698 (+ 724us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.281180 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.281188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.281194 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.281235 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.284281 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.285068 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.285073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.285075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.285202 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.285616 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.285620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.285622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.285909 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.286041 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.286465 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.286833 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":159,"server-negotiator.queue_time_us":735,"thread_start_us":432,"threads_started":1}
I20260812 00:37:26.288605 3501 log.cc:824] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:26.290258 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.271346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39778 (local address 127.7.244.125:44903)
0812 00:37:26.271529 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.271534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.271551 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:37:26.271690 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.271695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.271775 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.271872 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.273787 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.274852 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.276577 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.277221 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.277340 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.282727 (+ 5387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.282734 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.282739 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.282795 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.286301 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.288045 (+ 1744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.288051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.288053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.288215 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.288712 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.288715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.288717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.289018 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.289324 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.289662 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.289926 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:37:26.294852 3779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: Bootstrap starting.
I20260812 00:37:26.295457 3501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4: No bootstrap required, opened a new log
I20260812 00:37:26.296200 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:37:26.299052 3779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:26.300622 3779 log.cc:824] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:26.305760 3501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.306262 3501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:26.306404 3501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 937d9027ec00447eb8be755956a6d4f4, State: Initialized, Role: FOLLOWER
I20260812 00:37:26.307257 3501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.311311 3779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: No bootstrap required, opened a new log
I20260812 00:37:26.323269 3794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.302883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.7.244.126:42533)
0812 00:37:26.303064 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.303070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.303200 (+ 130us) server_negotiation.cc:407] Connection header received
0812 00:37:26.303518 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.303524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.303621 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.303701 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.306287 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.307544 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.310194 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.310862 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.310963 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.314885 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.314892 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.314896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.314951 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.318942 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.319950 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.319957 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.319960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.320437 (+ 477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.320936 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.320942 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.320945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.321689 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.321965 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.322593 (+ 628us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.322862 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:37:26.324544 3779 raft_consensus.cc:348] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.325191 3779 raft_consensus.cc:374] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:26.325379 3779 raft_consensus.cc:729] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e451067b33ac4e1cbeeef618adce627a, State: Initialized, Role: FOLLOWER
I20260812 00:37:26.326339 3779 consensus_queue.cc:260] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.327636 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.302445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39794 (local address 127.7.244.125:44903)
0812 00:37:26.302623 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.302629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.303780 (+ 1151us) server_negotiation.cc:407] Connection header received
0812 00:37:26.304048 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.304053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.304136 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.304214 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.306674 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.307794 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.310645 (+ 2851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.314918 (+ 4273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.315033 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.319499 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.319507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.319511 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.319560 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.322970 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.324245 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.324250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.324254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.324454 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.325693 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.325700 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.325703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.326079 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.326343 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.326915 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.327161 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:37:26.330948 3825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.331451 3825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:37:26.332703 3501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:26.333954 3826 sys_catalog.cc:455] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.334559 3826 sys_catalog.cc:458] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:37:26.335620 3779 sys_catalog.cc:564] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:26.341158 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.303948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56456 (local address 127.7.244.124:39273)
0812 00:37:26.304137 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.304143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.310364 (+ 6221us) server_negotiation.cc:407] Connection header received
0812 00:37:26.318362 (+ 7998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.318370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.318487 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.318573 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.327455 (+ 8882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.328517 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.330422 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.331083 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.331222 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.334674 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.334682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.334687 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.334735 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.338045 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.339003 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.339009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.339013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.339145 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.339559 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.339565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.339568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.339879 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.340142 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.340492 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.340723 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260812 00:37:26.349102 3402 tablet.cc:2369] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:37:26.359782 3864 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:37:26.360064 3864 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 00:37:26.369159 3859 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:37:26.369385 3859 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:37:26.359239 3354 master_runner.cc:186] Error getting master registration for 127.7.244.125:44903: OK, instance_id { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" instance_seqno: 1786495045658958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:37:26.374109 3354 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:37:26.364835 3655 master_runner.cc:186] Error getting master registration for 127.7.244.125:44903: OK, instance_id { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" instance_seqno: 1786495045658958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:37:26.377832 3655 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:37:26.391950 3639 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } attempt: 5
Aug 12 00:37:26 dist-test-slave-gjw7 krb5kdc[3333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495045, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
I20260812 00:37:26.411841 3646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.392849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:39273 (local address 127.0.0.1:56460)
0812 00:37:26.393184 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:26.393208 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:26.393435 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:26.393943 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:26.393948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:26.394571 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:26.394811 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.394827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.396536 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.396543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.398104 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:26.398126 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.399552 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.399558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:26.399781 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.400549 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:26.400574 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:26.403643 (+ 3069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:26.408367 (+ 4724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.408374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.408379 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.409015 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.409622 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:26.409628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:26.409632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:26.409832 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:26.410713 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:26.410719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:26.410917 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 00:37:26.411080 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 00:37:26.411430 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200}
I20260812 00:37:26.413692 3796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:26.392951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56460 (local address 127.7.244.124:39273)
0812 00:37:26.393185 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:26.393199 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:26.393362 (+ 163us) server_negotiation.cc:407] Connection header received
0812 00:37:26.393634 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:26.393642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:26.393738 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:26.393861 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:26.395033 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.396341 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.398347 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:26.399101 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:26.399262 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:26.403947 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:26.403959 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:26.403964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:26.404017 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:26.408112 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.409234 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.409241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.409245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.409465 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:26.410005 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:26.410011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:26.410015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:26.410423 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:26.410633 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:26.412990 (+ 2357us) server_negotiation.cc:299] Negotiation successful
0812 00:37:26.413266 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 00:37:26.419834 3639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: Bootstrap starting.
I20260812 00:37:26.424535 3639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:26.427264 3639 log.cc:824] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:26.431209 3639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: No bootstrap required, opened a new log
I20260812 00:37:26.444090 3639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.444602 3639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:26.444741 3639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 730ca0bf160347d085c7fb042d71e3fa, State: Initialized, Role: FOLLOWER
I20260812 00:37:26.445559 3639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.446859 3881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.447422 3881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:37:26.448249 3639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [sys.catalog]: configured and running, proceeding with master startup.
W20260812 00:37:26.448688 3703 tablet.cc:2369] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:37:26.470286 3512 master_runner.cc:263] No leader master found from master 730ca0bf160347d085c7fb042d71e3fa
I20260812 00:37:26.470461 3512 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 00:37:26.475461 3909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:37:26.475754 3909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 00:37:26.648357 3565 tablet.cc:2369] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:37:26.747763 3825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:37:26.748201 3825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.750304 3825 leader_election.cc:288] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 730ca0bf160347d085c7fb042d71e3fa (127.7.244.125:44903), e451067b33ac4e1cbeeef618adce627a (127.7.244.124:39273)
I20260812 00:37:26.751659 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "937d9027ec00447eb8be755956a6d4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e451067b33ac4e1cbeeef618adce627a" is_pre_election: true
I20260812 00:37:26.752403 3743 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 937d9027ec00447eb8be755956a6d4f4 in term 0.
I20260812 00:37:26.753549 3600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "937d9027ec00447eb8be755956a6d4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730ca0bf160347d085c7fb042d71e3fa" is_pre_election: true
I20260812 00:37:26.754252 3600 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 937d9027ec00447eb8be755956a6d4f4 in term 0.
I20260812 00:37:26.755102 3392 leader_election.cc:302] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 937d9027ec00447eb8be755956a6d4f4, e451067b33ac4e1cbeeef618adce627a; no voters:
I20260812 00:37:26.756368 3825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:37:26.756564 3825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:37:26.756692 3825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:26.760782 3825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.762392 3600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "937d9027ec00447eb8be755956a6d4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "730ca0bf160347d085c7fb042d71e3fa"
I20260812 00:37:26.762684 3600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:26.766472 3600 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937d9027ec00447eb8be755956a6d4f4 in term 1.
I20260812 00:37:26.767144 3392 leader_election.cc:302] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 730ca0bf160347d085c7fb042d71e3fa, 937d9027ec00447eb8be755956a6d4f4; no voters:
I20260812 00:37:26.778172 3825 leader_election.cc:288] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [CANDIDATE]: Term 1 election: Requested vote from peers 730ca0bf160347d085c7fb042d71e3fa (127.7.244.125:44903), e451067b33ac4e1cbeeef618adce627a (127.7.244.124:39273)
I20260812 00:37:26.778492 3825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:37:26.778674 3825 raft_consensus.cc:686] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [term 1 LEADER]: Becoming Leader. State: Replica: 937d9027ec00447eb8be755956a6d4f4, State: Running, Role: LEADER
I20260812 00:37:26.779335 3825 consensus_queue.cc:237] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } }
I20260812 00:37:26.791695 3743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "937d9027ec00447eb8be755956a6d4f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e451067b33ac4e1cbeeef618adce627a"
I20260812 00:37:26.792012 3743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:26.795809 3743 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 937d9027ec00447eb8be755956a6d4f4 in term 1.
I20260812 00:37:26.800344 3914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 937d9027ec00447eb8be755956a6d4f4. Latest consensus state: current_term: 1 leader_uuid: "937d9027ec00447eb8be755956a6d4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.800776 3914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:26.809324 3916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:37:26.811861 3916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:37:26.845389 3600 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [term 1 FOLLOWER]: Refusing update from remote peer 937d9027ec00447eb8be755956a6d4f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:37:26.846833 3743 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [term 1 FOLLOWER]: Refusing update from remote peer 937d9027ec00447eb8be755956a6d4f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:37:26.848014 3914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:37:26.848379 3825 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:37:26.863312 3881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 937d9027ec00447eb8be755956a6d4f4. Latest consensus state: current_term: 1 leader_uuid: "937d9027ec00447eb8be755956a6d4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.863679 3881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:37:26.876084 3920 mvcc.cc:205] Tried to move back new op lower bound from 7317483711832416256 to 7317483711625633792. Current Snapshot: MvccSnapshot[applied={T|T < 7317483711832416256}]
I20260812 00:37:26.878845 3881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "937d9027ec00447eb8be755956a6d4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.879174 3881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:37:26.881518 3914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "937d9027ec00447eb8be755956a6d4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.881834 3914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:26.888710 3927 sys_catalog.cc:455] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 937d9027ec00447eb8be755956a6d4f4. Latest consensus state: current_term: 1 leader_uuid: "937d9027ec00447eb8be755956a6d4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.890601 3927 sys_catalog.cc:458] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:37:26.891492 3916 catalog_manager.cc:1347] Generated new cluster ID: df67bf7e98dc4ce4a1105154132b8212
I20260812 00:37:26.891624 3916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:37:26.892503 3914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "937d9027ec00447eb8be755956a6d4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.892830 3914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:26.910000 3927 sys_catalog.cc:455] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "937d9027ec00447eb8be755956a6d4f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "937d9027ec00447eb8be755956a6d4f4" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42533 } } peers { permanent_uuid: "730ca0bf160347d085c7fb042d71e3fa" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 44903 } } peers { permanent_uuid: "e451067b33ac4e1cbeeef618adce627a" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 39273 } } }
I20260812 00:37:26.910339 3927 sys_catalog.cc:458] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:37:26.931098 3916 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:37:26.933324 3916 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:37:26.945029 3916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 937d9027ec00447eb8be755956a6d4f4: Generated new TSK 0
I20260812 00:37:26.946357 3916 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:37:27.371441 3859 catalog_manager.cc:1259] Loaded cluster ID: df67bf7e98dc4ce4a1105154132b8212
I20260812 00:37:27.371557 3859 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: loading cluster ID for follower catalog manager: success
I20260812 00:37:27.374464 3859 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: acquiring CA information for follower catalog manager: success
I20260812 00:37:27.376577 3859 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e451067b33ac4e1cbeeef618adce627a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:37:27.478183 3909 catalog_manager.cc:1259] Loaded cluster ID: df67bf7e98dc4ce4a1105154132b8212
I20260812 00:37:27.478325 3909 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: loading cluster ID for follower catalog manager: success
I20260812 00:37:27.481112 3909 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: acquiring CA information for follower catalog manager: success
I20260812 00:37:27.483284 3909 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 730ca0bf160347d085c7fb042d71e3fa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:37:27.524350 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 3354
I20260812 00:37:27.525540 3492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:27.637928 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 3354
I20260812 00:37:27.673888 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 3512
I20260812 00:37:27.675168 3627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:27.782200 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 3512
I20260812 00:37:27.806814 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 3655
I20260812 00:37:27.808041 3772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:27.920445 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 3655
2026-08-12T00:37:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4910 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:37:28 dist-test-slave-gjw7 krb5kdc[3957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:37:28 dist-test-slave-gjw7 krb5kdc[3957](info): set up 2 sockets
Aug 12 00:37:28 dist-test-slave-gjw7 krb5kdc[3957](info): commencing operation
krb5kdc: starting...
W20260812 00:37:29.256426 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:37:29 dist-test-slave-gjw7 krb5kdc[3957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:37:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:37:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:37:29.554177 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41855
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:41855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:29.920351 3986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:29.920728 3986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:29.920881 3986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:29.928599 3986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:29.928754 3986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:29.928831 3986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:29.928918 3986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:29.928985 3986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:29.939689 3986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:41855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:29.941461 3986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:29.943034 3986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:29.956292 3994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:29.956218 3993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:29.956238 3997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:29.957257 3986 server_base.cc:1034] running on GCE node
I20260812 00:37:29.958218 3986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:29.959865 3986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:29.961138 3986 hybrid_clock.cc:648] HybridClock initialized: now 1786495049961078 us; error 49 us; skew 500 ppm
I20260812 00:37:29.961615 3986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:29 dist-test-slave-gjw7 krb5kdc[3957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495049, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:29.965117 3986 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:37:29.967016 3986 webserver.cc:460] Webserver started at http://127.7.244.126:45685/ using document root <none> and password file <none>
I20260812 00:37:29.967859 3986 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:29.968004 3986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:29.968369 3986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:29.971740 3986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "a815054721844d40aa84624044a409f1"
format_stamp: "Formatted at 2026-08-12 00:37:29 on dist-test-slave-gjw7"
server_key: "a3172f08fbf5c8be39f43560ad801d17"
server_key_iv: "b077fd1fa9c38b2647c344e0bb3e4545"
server_key_version: "encryptionkey@0"
I20260812 00:37:29.972612 3986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "a815054721844d40aa84624044a409f1"
format_stamp: "Formatted at 2026-08-12 00:37:29 on dist-test-slave-gjw7"
server_key: "a3172f08fbf5c8be39f43560ad801d17"
server_key_iv: "b077fd1fa9c38b2647c344e0bb3e4545"
server_key_version: "encryptionkey@0"
I20260812 00:37:29.978857 3986 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260812 00:37:29.982867 4007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:29.984385 3986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260812 00:37:29.984588 3986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "a815054721844d40aa84624044a409f1"
format_stamp: "Formatted at 2026-08-12 00:37:29 on dist-test-slave-gjw7"
server_key: "a3172f08fbf5c8be39f43560ad801d17"
server_key_iv: "b077fd1fa9c38b2647c344e0bb3e4545"
server_key_version: "encryptionkey@0"
I20260812 00:37:29.984804 3986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:30.000527 3986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:30.007303 3986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:30.007670 3986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:30.028993 3986 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:41855
I20260812 00:37:30.028999 4118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:41855 every 8 connection(s)
I20260812 00:37:30.030658 3986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:37:30.034910 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 3986
I20260812 00:37:30.035166 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:37:30.035347 4122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:30.035905 8145 external_mini_cluster.cc:1428] Setting key 893d0522d1dfe29413de1f4a87aa373d
I20260812 00:37:30.051326 4122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1: Bootstrap starting.
I20260812 00:37:30.055332 4122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:30.056915 4122 log.cc:824] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:37:30 dist-test-slave-gjw7 krb5kdc[3957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:30.060403 4122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1: No bootstrap required, opened a new log
I20260812 00:37:30.067461 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:30.037978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35820 (local address 127.7.244.126:41855)
0812 00:37:30.039029 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:30.039042 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:30.039095 (+ 53us) server_negotiation.cc:407] Connection header received
0812 00:37:30.041023 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:30.041053 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:30.041640 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:30.042064 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:30.044184 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:30.045707 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:30.047814 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:30.048749 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:30.048942 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:30.058926 (+ 9984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:30.058936 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:30.058956 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:30.059001 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:30.062265 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:30.063300 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:30.063310 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:30.063318 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:30.063549 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:30.064047 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:30.064052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:30.064055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:30.064733 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:30.065114 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:30.065698 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 00:37:30.066227 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":825,"thread_start_us":539,"threads_started":1}
I20260812 00:37:30.069242 4122 raft_consensus.cc:348] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a815054721844d40aa84624044a409f1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41855 } }
I20260812 00:37:30.069736 4122 raft_consensus.cc:374] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:30.069854 4122 raft_consensus.cc:729] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a815054721844d40aa84624044a409f1, State: Initialized, Role: FOLLOWER
I20260812 00:37:30.070551 4122 consensus_queue.cc:260] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a815054721844d40aa84624044a409f1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41855 } }
I20260812 00:37:30.070870 4122 raft_consensus.cc:388] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:37:30.071030 4122 raft_consensus.cc:482] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:37:30.071193 4122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:30.072727 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:37:30.074185 4122 raft_consensus.cc:504] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a815054721844d40aa84624044a409f1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41855 } }
I20260812 00:37:30.074803 4122 leader_election.cc:302] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a815054721844d40aa84624044a409f1; no voters:
I20260812 00:37:30.075573 4122 leader_election.cc:288] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:37:30.075714 4132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:37:30.077015 4132 raft_consensus.cc:686] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [term 1 LEADER]: Becoming Leader. State: Replica: a815054721844d40aa84624044a409f1, State: Running, Role: LEADER
I20260812 00:37:30.077606 4132 consensus_queue.cc:237] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a815054721844d40aa84624044a409f1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41855 } }
I20260812 00:37:30.078161 4122 sys_catalog.cc:564] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:30.080494 4138 sys_catalog.cc:455] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a815054721844d40aa84624044a409f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a815054721844d40aa84624044a409f1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41855 } } }
I20260812 00:37:30.080739 4139 sys_catalog.cc:455] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a815054721844d40aa84624044a409f1. Latest consensus state: current_term: 1 leader_uuid: "a815054721844d40aa84624044a409f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a815054721844d40aa84624044a409f1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41855 } } }
I20260812 00:37:30.081316 4139 sys_catalog.cc:458] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:30.081312 4138 sys_catalog.cc:458] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:30.082371 4144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:37:30.087530 4144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:37:30.097674 4144 catalog_manager.cc:1347] Generated new cluster ID: 830d0fc3bcf64b92a7bf57b1117e85f9
I20260812 00:37:30.097822 4144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:37:30.123440 4144 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:37:30.124948 4144 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:37:30.137080 4144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a815054721844d40aa84624044a409f1: Generated new TSK 0
I20260812 00:37:30.138156 4144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:37:30.186532 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 3986
I20260812 00:37:30.187916 4110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:30.286734 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 3986
2026-08-12T00:37:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2368 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:37:30 dist-test-slave-gjw7 krb5kdc[4176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:37:30 dist-test-slave-gjw7 krb5kdc[4176](info): set up 2 sockets
Aug 12 00:37:30 dist-test-slave-gjw7 krb5kdc[4176](info): commencing operation
krb5kdc: starting...
W20260812 00:37:31.628177 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.006s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:37:31 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:37:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:37:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:37:31.923072 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39087
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:39087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:32.288589 4217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:32.288950 4217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:32.289075 4217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:32.296824 4217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:32.296976 4217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:32.297036 4217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:32.297120 4217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:32.297180 4217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:32.307776 4217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:39087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:32.309268 4217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:32.310740 4217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:32.323331 4226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:32.323395 4227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:32.323992 4232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:32.324142 4217 server_base.cc:1034] running on GCE node
I20260812 00:37:32.325083 4217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:32.326784 4217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:32.328037 4217 hybrid_clock.cc:648] HybridClock initialized: now 1786495052327977 us; error 53 us; skew 500 ppm
I20260812 00:37:32.328449 4217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:32 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495052, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:32.332098 4217 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:37:32.334030 4217 webserver.cc:460] Webserver started at http://127.7.244.126:33239/ using document root <none> and password file <none>
I20260812 00:37:32.335152 4217 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:32.335330 4217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:32.335773 4217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:32.339666 4217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "3bf3cd2327f74025a7908a9a97c9fa4d"
format_stamp: "Formatted at 2026-08-12 00:37:32 on dist-test-slave-gjw7"
server_key: "c5f067dd214f90f68d7b609fe3949e9c"
server_key_iv: "968a32773a0b19c75cd8273877b9e8fc"
server_key_version: "encryptionkey@0"
I20260812 00:37:32.340626 4217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "3bf3cd2327f74025a7908a9a97c9fa4d"
format_stamp: "Formatted at 2026-08-12 00:37:32 on dist-test-slave-gjw7"
server_key: "c5f067dd214f90f68d7b609fe3949e9c"
server_key_iv: "968a32773a0b19c75cd8273877b9e8fc"
server_key_version: "encryptionkey@0"
I20260812 00:37:32.346891 4217 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 00:37:32.351603 4243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:32.353231 4217 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260812 00:37:32.353514 4217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "3bf3cd2327f74025a7908a9a97c9fa4d"
format_stamp: "Formatted at 2026-08-12 00:37:32 on dist-test-slave-gjw7"
server_key: "c5f067dd214f90f68d7b609fe3949e9c"
server_key_iv: "968a32773a0b19c75cd8273877b9e8fc"
server_key_version: "encryptionkey@0"
I20260812 00:37:32.353749 4217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:32.390192 4217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:32.395128 4217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:32.395416 4217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:32.414819 4217 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:39087
I20260812 00:37:32.414850 4340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:39087 every 8 connection(s)
I20260812 00:37:32.416404 4217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:37:32.420756 4342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:32.423480 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 4217
I20260812 00:37:32.423725 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:37:32.424533 8145 external_mini_cluster.cc:1428] Setting key efda4df70b65badca7514ab5c9beb4b6
I20260812 00:37:32.434702 4342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Bootstrap starting.
I20260812 00:37:32.439653 4342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Neither blocks nor log segments found. Creating new log.
Aug 12 00:37:32 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:32.441457 4342 log.cc:824] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:32.444964 4342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: No bootstrap required, opened a new log
I20260812 00:37:32.451401 4349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:32.426917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.7.244.126:39087)
0812 00:37:32.427767 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:32.427780 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:32.427824 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:37:32.429643 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:32.429671 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:32.430132 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:32.430515 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:32.432450 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:32.434013 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:32.435972 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:32.436917 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:32.437075 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:32.442772 (+ 5697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:32.442782 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:32.442803 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:32.442849 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:32.446283 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:32.447263 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:32.447273 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:32.447285 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:32.447506 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:32.447997 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:32.448002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:32.448004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:32.448693 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:32.449034 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:32.449661 (+ 627us) server_negotiation.cc:299] Negotiation successful
0812 00:37:32.450188 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":388,"threads_started":1}
I20260812 00:37:32.453898 4342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:32.454470 4342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:32.454600 4342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf3cd2327f74025a7908a9a97c9fa4d, State: Initialized, Role: FOLLOWER
I20260812 00:37:32.455322 4342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:32.455579 4342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:37:32.455719 4342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:37:32.455878 4342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:32.458437 4342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:32.459147 4342 leader_election.cc:302] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bf3cd2327f74025a7908a9a97c9fa4d; no voters:
I20260812 00:37:32.459893 4342 leader_election.cc:288] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:37:32.460055 4354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:37:32.461162 4354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 1 LEADER]: Becoming Leader. State: Replica: 3bf3cd2327f74025a7908a9a97c9fa4d, State: Running, Role: LEADER
I20260812 00:37:32.461882 4354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:32.462522 4342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:32.465029 4356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bf3cd2327f74025a7908a9a97c9fa4d. Latest consensus state: current_term: 1 leader_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } } }
I20260812 00:37:32.466543 4356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:32.465055 4355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } } }
I20260812 00:37:32.466775 4355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:32.468202 4375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:37:32.474165 4375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:37:32.485561 4375 catalog_manager.cc:1347] Generated new cluster ID: 9e0704bf31b1498ba26eb5a096db3ebb
I20260812 00:37:32.485697 4375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:37:32.504659 4375 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:37:32.506217 4375 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:37:32.520335 4375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Generated new TSK 0
I20260812 00:37:32.521373 4375 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:37:32.605487 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39087
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:32.979061 4398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:32.979413 4398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:32.979560 4398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:32.987334 4398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:32.987494 4398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:32.987661 4398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:37:32.998145 4398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:32.999594 4398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:33.001044 4398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:33.015466 4412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:33.015666 4405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:33.016026 4406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:33.016503 4398 server_base.cc:1034] running on GCE node
I20260812 00:37:33.017242 4398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:33.018466 4398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:33.019726 4398 hybrid_clock.cc:648] HybridClock initialized: now 1786495053019670 us; error 57 us; skew 500 ppm
I20260812 00:37:33.020114 4398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:33 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495053, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:33.023685 4398 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:37:33.025499 4398 webserver.cc:460] Webserver started at http://127.7.244.65:45283/ using document root <none> and password file <none>
I20260812 00:37:33.026326 4398 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:33.026469 4398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:33.026806 4398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:33.030292 4398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "858abe7d735d4151b42343a729fc3f9e"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "db46d8b49b52b40e59d28053c71a3781"
server_key_iv: "68e8784f51c75408006eea9c2fa83d8a"
server_key_version: "encryptionkey@0"
I20260812 00:37:33.031216 4398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "858abe7d735d4151b42343a729fc3f9e"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "db46d8b49b52b40e59d28053c71a3781"
server_key_iv: "68e8784f51c75408006eea9c2fa83d8a"
server_key_version: "encryptionkey@0"
I20260812 00:37:33.037675 4398 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 00:37:33.041812 4425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:33.043766 4398 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 00:37:33.044049 4398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "858abe7d735d4151b42343a729fc3f9e"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "db46d8b49b52b40e59d28053c71a3781"
server_key_iv: "68e8784f51c75408006eea9c2fa83d8a"
server_key_version: "encryptionkey@0"
I20260812 00:37:33.044371 4398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:33.083699 4398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:33.088389 4398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:33.088675 4398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:33.089885 4398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:33.091502 4398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:33.091591 4398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:33.091739 4398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:33.091796 4398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:33.128192 4398 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:38521
I20260812 00:37:33.128232 4667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:38521 every 8 connection(s)
I20260812 00:37:33.129616 4398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:37:33.131376 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 4398
I20260812 00:37:33.131595 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:37:33.132448 8145 external_mini_cluster.cc:1428] Setting key f16cf29eb1789e2473f8aa79ed301dab
Aug 12 00:37:33 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495053, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:33.152733 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:33.133534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:41845 (local address 127.7.244.126:39087)
0812 00:37:33.134101 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:33.134107 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:33.134948 (+ 841us) server_negotiation.cc:407] Connection header received
0812 00:37:33.137100 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:33.137107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:33.137224 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:33.137352 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:33.139136 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:33.139887 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:33.141679 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:33.142117 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:33.142242 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:33.146220 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:33.146226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:33.146229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:33.146268 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:33.148864 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:33.149900 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:33.149907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:33.149910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:33.150116 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:33.150676 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:33.150681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:33.150684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:33.151056 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:33.151250 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:33.152084 (+ 834us) server_negotiation.cc:299] Negotiation successful
0812 00:37:33.152386 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":306,"threads_started":1}
I20260812 00:37:33.153743 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:33.133469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:41845)
0812 00:37:33.134744 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:33.134801 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:33.136753 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:33.137660 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:33.137670 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:33.138261 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:33.138912 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:33.138933 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:33.140078 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:33.140082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:33.141494 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:33.141511 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:33.142411 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:33.142420 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:33.142658 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:33.143371 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:33.143401 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:33.145931 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:33.149044 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:33.149055 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:33.149071 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:33.149704 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:33.150283 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:33.150289 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:33.150293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:33.150537 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:33.151173 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:33.151179 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:33.151633 (+ 454us) client_negotiation.cc:769] Sending connection context
0812 00:37:33.151897 (+ 264us) client_negotiation.cc:240] Negotiation successful
0812 00:37:33.152464 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":667,"threads_started":1}
I20260812 00:37:33.155562 4668 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39087
I20260812 00:37:33.155901 4668 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:33.156731 4668 heartbeater.cc:511] Master 127.7.244.126:39087 requested a full tablet report, sending...
I20260812 00:37:33.159322 4272 ts_manager.cc:194] Registered new tserver with Master: 858abe7d735d4151b42343a729fc3f9e (127.7.244.65:38521)
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
I20260812 00:37:33.160945 4272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:41845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:37:33.196847 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39087
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:33.564666 4679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:33.565012 4679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:33.565114 4679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:33.572669 4679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:33.572808 4679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:33.572932 4679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:37:33.583103 4679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:33.584574 4679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:33.586061 4679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:33.600425 4685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:33.600333 4688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:33.600310 4686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:33.601394 4679 server_base.cc:1034] running on GCE node
I20260812 00:37:33.602126 4679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:33.603214 4679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:33.604458 4679 hybrid_clock.cc:648] HybridClock initialized: now 1786495053604391 us; error 70 us; skew 500 ppm
I20260812 00:37:33.604803 4679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:33 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495053, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:33.608283 4679 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:37:33.610023 4679 webserver.cc:460] Webserver started at http://127.7.244.66:45335/ using document root <none> and password file <none>
I20260812 00:37:33.610965 4679 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:33.611109 4679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:33.611445 4679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:33.614763 4679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "482432aabf084ff5828eaa5f4915240a"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "1677ef2d9f1b24495347bbd920bddf2b"
server_key_iv: "c0966c940467c4cf0adfff2a297bb747"
server_key_version: "encryptionkey@0"
I20260812 00:37:33.615732 4679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "482432aabf084ff5828eaa5f4915240a"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "1677ef2d9f1b24495347bbd920bddf2b"
server_key_iv: "c0966c940467c4cf0adfff2a297bb747"
server_key_version: "encryptionkey@0"
I20260812 00:37:33.621719 4679 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 00:37:33.625787 4702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:33.627351 4679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:37:33.627548 4679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "482432aabf084ff5828eaa5f4915240a"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "1677ef2d9f1b24495347bbd920bddf2b"
server_key_iv: "c0966c940467c4cf0adfff2a297bb747"
server_key_version: "encryptionkey@0"
I20260812 00:37:33.627766 4679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:33.648694 4679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:33.653826 4679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:33.654148 4679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:33.655329 4679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:33.657094 4679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:33.657203 4679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:33.657353 4679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:33.657413 4679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 00:37:33.690526 4679 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:36563
I20260812 00:37:33.690641 4950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:36563 every 8 connection(s)
I20260812 00:37:33.691990 4679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:37:33.697610 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 4679
I20260812 00:37:33.697851 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:37:33.698416 8145 external_mini_cluster.cc:1428] Setting key 3c5dc507b5310e63796d91f30a97f501
Aug 12 00:37:33 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495053, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:33.716639 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:33.694919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:43213 (local address 127.7.244.126:39087)
0812 00:37:33.695409 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:33.695414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:33.696495 (+ 1081us) server_negotiation.cc:407] Connection header received
0812 00:37:33.699004 (+ 2509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:33.699010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:33.699113 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:33.699248 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:33.701344 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:33.702126 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:33.704603 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:33.705069 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:33.705212 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:33.709984 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:33.709990 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:33.709994 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:33.710036 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:33.713043 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:33.714065 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:33.714072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:33.714075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:33.714255 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:33.714727 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:33.714733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:33.714736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:33.715105 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:33.715265 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:33.716005 (+ 740us) server_negotiation.cc:299] Negotiation successful
0812 00:37:33.716245 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":249,"threads_started":1}
I20260812 00:37:33.717588 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:33.695421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:43213)
0812 00:37:33.696259 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:33.696303 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:33.698605 (+ 2302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:33.699574 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:33.699587 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:33.700213 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:33.701098 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:33.701133 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:33.702340 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:33.702347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:33.704354 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:33.704378 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:33.705406 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:33.705416 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:33.705730 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:33.706601 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:33.706639 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:33.709647 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:33.713282 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:33.713330 (+ 48us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:33.713353 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:33.713896 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:33.714376 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:33.714379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:33.714382 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:33.714602 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:33.715220 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:33.715226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:33.715592 (+ 366us) client_negotiation.cc:769] Sending connection context
0812 00:37:33.715831 (+ 239us) client_negotiation.cc:240] Negotiation successful
0812 00:37:33.716307 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":318,"threads_started":1}
I20260812 00:37:33.719300 4952 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39087
I20260812 00:37:33.719699 4952 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:33.720659 4952 heartbeater.cc:511] Master 127.7.244.126:39087 requested a full tablet report, sending...
I20260812 00:37:33.722616 4272 ts_manager.cc:194] Registered new tserver with Master: 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563)
I20260812 00:37:33.723516 4272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:43213
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:37:33.764995 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39087
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:34.123905 4959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:34.124215 4959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:34.124315 4959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:34.132313 4959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:34.132445 4959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:34.132601 4959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:37:34.143257 4959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:34.144743 4959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:34.146238 4959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:34.160419 4970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:34.160444 4977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:34.160419 4972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:34.161449 4959 server_base.cc:1034] running on GCE node
I20260812 00:37:34.162138 4959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:34.163254 4959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:34.164481 4959 hybrid_clock.cc:648] HybridClock initialized: now 1786495054164424 us; error 50 us; skew 500 ppm
I20260812 00:37:34.164832 4959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:37:34.164975 4668 heartbeater.cc:503] Master 127.7.244.126:39087 was elected leader, sending a full tablet report...
Aug 12 00:37:34 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:34.168401 4959 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:37:34.170177 4959 webserver.cc:460] Webserver started at http://127.7.244.67:33919/ using document root <none> and password file <none>
I20260812 00:37:34.171002 4959 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:34.171144 4959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:34.171487 4959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:34.175284 4959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "07194dc481514a97835962d7f46b7d92"
format_stamp: "Formatted at 2026-08-12 00:37:34 on dist-test-slave-gjw7"
server_key: "0600622515e311130d87b31976d809a1"
server_key_iv: "4a9415f965500195c465aaab0c24bb27"
server_key_version: "encryptionkey@0"
I20260812 00:37:34.176265 4959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "07194dc481514a97835962d7f46b7d92"
format_stamp: "Formatted at 2026-08-12 00:37:34 on dist-test-slave-gjw7"
server_key: "0600622515e311130d87b31976d809a1"
server_key_iv: "4a9415f965500195c465aaab0c24bb27"
server_key_version: "encryptionkey@0"
I20260812 00:37:34.182507 4959 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 00:37:34.186844 4991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:34.188583 4959 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260812 00:37:34.188795 4959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "07194dc481514a97835962d7f46b7d92"
format_stamp: "Formatted at 2026-08-12 00:37:34 on dist-test-slave-gjw7"
server_key: "0600622515e311130d87b31976d809a1"
server_key_iv: "4a9415f965500195c465aaab0c24bb27"
server_key_version: "encryptionkey@0"
I20260812 00:37:34.189036 4959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:34.212878 4959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:34.217808 4959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:34.218114 4959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:34.219226 4959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:34.220733 4959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:34.220827 4959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:34.220960 4959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:34.221016 4959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:34.254241 4959 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:36345
I20260812 00:37:34.254287 5198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:36345 every 8 connection(s)
I20260812 00:37:34.255627 4959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:37:34.265122 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 4959
I20260812 00:37:34.265357 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:37:34.265959 8145 external_mini_cluster.cc:1428] Setting key 2c2a480f3fc93b3927ad99335cf2238b
Aug 12 00:37:34 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495054, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:34.277165 5201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.258436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:51503 (local address 127.7.244.126:39087)
0812 00:37:34.258947 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.258954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.259887 (+ 933us) server_negotiation.cc:407] Connection header received
0812 00:37:34.261828 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.261834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.261925 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.262026 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:34.263667 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.264492 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.266802 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.267329 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.267454 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.271280 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:34.271286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:34.271289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:34.271327 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:34.273934 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:34.274805 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:34.274810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:34.274812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:34.274955 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:34.275424 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:34.275429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:34.275431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:34.275736 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:34.275893 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.276658 (+ 765us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.276875 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":231,"threads_started":1}
I20260812 00:37:34.278108 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.259003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:51503)
0812 00:37:34.259712 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:34.259761 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:34.261529 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:34.262288 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:34.262297 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:34.262864 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:34.263459 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.263481 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.264661 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.264667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.266637 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.266656 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.267576 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.267584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.267846 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.268616 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:34.268642 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:34.271054 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:34.274090 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:34.274098 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:34.274115 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:34.274620 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:34.275062 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:34.275066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:34.275068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:34.275271 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:34.275848 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:34.275855 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:34.276246 (+ 391us) client_negotiation.cc:769] Sending connection context
0812 00:37:34.276487 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 00:37:34.276922 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":270,"threads_started":1}
I20260812 00:37:34.279685 5199 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39087
I20260812 00:37:34.280036 5199 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:34.280972 5199 heartbeater.cc:511] Master 127.7.244.126:39087 requested a full tablet report, sending...
I20260812 00:37:34.282680 4272 ts_manager.cc:194] Registered new tserver with Master: 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345)
I20260812 00:37:34.283476 4272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:51503
I20260812 00:37:34.295359 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:37:34.313814 5201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.300283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.7.244.126:39087)
0812 00:37:34.300466 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.300472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.300861 (+ 389us) server_negotiation.cc:407] Connection header received
0812 00:37:34.301041 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.301045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.301116 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.301214 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:34.302805 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.303597 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.305087 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.305608 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.305715 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.308385 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:34.308391 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:34.308394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:34.308449 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:34.311130 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:34.311994 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:34.311998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:34.312000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:34.312125 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:34.312475 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:34.312478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:34.312480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:34.312811 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:34.312997 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.313337 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.313563 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:37:34.318297 4272 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:37:34.325788 4272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:37:34.369271 5223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.340463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.7.244.65:38521)
0812 00:37:34.343132 (+ 2669us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.343145 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.360005 (+ 16860us) server_negotiation.cc:407] Connection header received
0812 00:37:34.360397 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.360408 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.360706 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.360910 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:34.361856 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.362996 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.366128 (+ 3132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.367690 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.367868 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.368163 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.368618 (+ 455us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.368826 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2344,"thread_start_us":2055,"threads_started":1}
I20260812 00:37:34.369583 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.340039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41584)
0812 00:37:34.359818 (+ 19779us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:34.359844 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:34.360155 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:34.361226 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:34.361230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:34.361261 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:34.361684 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.361694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.363424 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.363429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.365924 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.365945 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.367986 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.367995 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.368264 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.368340 (+ 76us) client_negotiation.cc:769] Sending connection context
0812 00:37:34.368510 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 00:37:34.368783 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19571,"mutex_wait_us":55,"thread_start_us":19706,"threads_started":1}
I20260812 00:37:34.370316 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.340983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52494 (local address 127.7.244.67:36345)
0812 00:37:34.359347 (+ 18364us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.359356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.359399 (+ 43us) server_negotiation.cc:407] Connection header received
0812 00:37:34.359522 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.359531 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.359882 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.360173 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:34.361045 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.362974 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.366668 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.367890 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.368042 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.368327 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.369089 (+ 762us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.369944 (+ 855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18181,"thread_start_us":18022,"threads_started":1}
I20260812 00:37:34.370508 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.339950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:36345 (local address 127.0.0.1:52494)
0812 00:37:34.344592 (+ 4642us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:34.344656 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:34.344882 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:34.360200 (+ 15318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:34.360210 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:34.360242 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:34.360745 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.360762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.363182 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.363189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.366457 (+ 3268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.366468 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.368509 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.368517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.368741 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.368822 (+ 81us) client_negotiation.cc:769] Sending connection context
0812 00:37:34.368924 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 00:37:34.370016 (+ 1092us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4319,"mutex_wait_us":1013,"thread_start_us":19418,"threads_started":1}
I20260812 00:37:34.371395 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.341811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:36563 (local address 127.0.0.1:51850)
0812 00:37:34.360065 (+ 18254us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:34.360085 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:34.360211 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:34.360937 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:34.360941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:34.360968 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:34.361420 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.361434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.363533 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.363540 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.366394 (+ 2854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.366422 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.368584 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.368592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.368812 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.368846 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 00:37:34.368931 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 00:37:34.369117 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18120,"mutex_wait_us":107,"thread_start_us":355,"threads_started":1}
I20260812 00:37:34.371611 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.341076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.7.244.66:36563)
0812 00:37:34.358555 (+ 17479us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.358566 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.360364 (+ 1798us) server_negotiation.cc:407] Connection header received
0812 00:37:34.360499 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.360507 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.360729 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.360911 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:34.361638 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.363334 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.366627 (+ 3293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.368111 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.368255 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.368410 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.370952 (+ 2542us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.371282 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17214,"thread_start_us":17454,"threads_started":1}
I20260812 00:37:34.374135 4526 tablet_service.cc:1467] Processing CreateTablet for tablet a155f6b67b9d45c48977035f6ee75904 (DEFAULT_TABLE table=test-table [id=50d30b5f283441e09340791e2bab4c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:34.374855 5081 tablet_service.cc:1467] Processing CreateTablet for tablet a155f6b67b9d45c48977035f6ee75904 (DEFAULT_TABLE table=test-table [id=50d30b5f283441e09340791e2bab4c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:34.375545 4526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a155f6b67b9d45c48977035f6ee75904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:34.375464 4811 tablet_service.cc:1467] Processing CreateTablet for tablet a155f6b67b9d45c48977035f6ee75904 (DEFAULT_TABLE table=test-table [id=50d30b5f283441e09340791e2bab4c0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:34.376502 5081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a155f6b67b9d45c48977035f6ee75904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:34.376935 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a155f6b67b9d45c48977035f6ee75904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:34.388803 5229 tablet_bootstrap.cc:492] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Bootstrap starting.
I20260812 00:37:34.392704 5229 tablet_bootstrap.cc:654] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:34.392791 5231 tablet_bootstrap.cc:492] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Bootstrap starting.
I20260812 00:37:34.394243 5229 log.cc:824] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:34.395495 5232 tablet_bootstrap.cc:492] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Bootstrap starting.
I20260812 00:37:34.396880 5231 tablet_bootstrap.cc:654] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:34.398404 5231 log.cc:824] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:34.399402 5229 tablet_bootstrap.cc:492] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: No bootstrap required, opened a new log
I20260812 00:37:34.399677 5229 ts_tablet_manager.cc:1397] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260812 00:37:34.401875 5231 tablet_bootstrap.cc:492] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: No bootstrap required, opened a new log
I20260812 00:37:34.402195 5231 ts_tablet_manager.cc:1397] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 00:37:34.403417 5232 tablet_bootstrap.cc:654] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:34.404541 5232 log.cc:824] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:34.407481 5232 tablet_bootstrap.cc:492] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: No bootstrap required, opened a new log
I20260812 00:37:34.407773 5232 ts_tablet_manager.cc:1397] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260812 00:37:34.410600 5229 raft_consensus.cc:348] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.411175 5229 raft_consensus.cc:374] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:34.411281 5229 raft_consensus.cc:729] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858abe7d735d4151b42343a729fc3f9e, State: Initialized, Role: FOLLOWER
I20260812 00:37:34.412066 5229 consensus_queue.cc:260] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.412726 5231 raft_consensus.cc:348] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.413257 5231 raft_consensus.cc:374] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:34.413395 5231 raft_consensus.cc:729] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482432aabf084ff5828eaa5f4915240a, State: Initialized, Role: FOLLOWER
I20260812 00:37:34.414184 5231 consensus_queue.cc:260] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.414557 5229 ts_tablet_manager.cc:1428] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260812 00:37:34.415494 4952 heartbeater.cc:503] Master 127.7.244.126:39087 was elected leader, sending a full tablet report...
I20260812 00:37:34.416239 5231 ts_tablet_manager.cc:1428] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260812 00:37:34.419835 5232 raft_consensus.cc:348] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.420439 5232 raft_consensus.cc:374] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:34.420578 5232 raft_consensus.cc:729] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07194dc481514a97835962d7f46b7d92, State: Initialized, Role: FOLLOWER
I20260812 00:37:34.421497 5232 consensus_queue.cc:260] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.422744 5199 heartbeater.cc:503] Master 127.7.244.126:39087 was elected leader, sending a full tablet report...
I20260812 00:37:34.423349 5232 ts_tablet_manager.cc:1428] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260812 00:37:34.444402 4953 tablet.cc:2369] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:37:34.507475 5200 tablet.cc:2369] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:37:34.632668 4670 tablet.cc:2369] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:37:34.743713 5241 raft_consensus.cc:482] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:37:34.744062 5241 raft_consensus.cc:504] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.745985 5241 leader_election.cc:288] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345), 858abe7d735d4151b42343a729fc3f9e (127.7.244.65:38521)
I20260812 00:37:34.753521 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.746427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:52667 (local address 127.7.244.67:36345)
0812 00:37:34.746814 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.746823 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.747633 (+ 810us) server_negotiation.cc:407] Connection header received
0812 00:37:34.747796 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.747800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.747864 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.747957 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:34.748668 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.749462 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.751624 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.752493 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.752678 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.752758 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.753087 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.753226 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:37:34.753546 5223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.746612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:50751 (local address 127.7.244.65:38521)
0812 00:37:34.746886 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.746893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.747567 (+ 674us) server_negotiation.cc:407] Connection header received
0812 00:37:34.747753 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.747759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.747828 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.747897 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:34.748576 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.749402 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.751606 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.752523 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.752677 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.752758 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.753135 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.753275 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260812 00:37:34.753944 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.746424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:36345 (local address 127.7.244.66:52667)
0812 00:37:34.747488 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:34.747504 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:34.747651 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:34.747994 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:34.747998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:34.748029 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:34.748478 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.748491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.749610 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.749614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.751444 (+ 1830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.751461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.752755 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.752762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.752914 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.752941 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 00:37:34.753022 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 00:37:34.753162 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"mutex_wait_us":57,"thread_start_us":654,"threads_started":1}
I20260812 00:37:34.753988 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.746397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.7.244.66:50751)
0812 00:37:34.747417 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:34.747434 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:34.747599 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:34.747957 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:34.747962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:34.747998 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:34.748411 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.748423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.749550 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.749556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.751428 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:34.751445 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.752814 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.752820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:34.752964 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.752987 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 00:37:34.753083 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 00:37:34.753222 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":122,"thread_start_us":663,"threads_started":1}
I20260812 00:37:34.755069 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a155f6b67b9d45c48977035f6ee75904" candidate_uuid: "482432aabf084ff5828eaa5f4915240a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07194dc481514a97835962d7f46b7d92" is_pre_election: true
I20260812 00:37:34.755069 4581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a155f6b67b9d45c48977035f6ee75904" candidate_uuid: "482432aabf084ff5828eaa5f4915240a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858abe7d735d4151b42343a729fc3f9e" is_pre_election: true
I20260812 00:37:34.755656 5116 raft_consensus.cc:2454] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482432aabf084ff5828eaa5f4915240a in term 0.
I20260812 00:37:34.755654 4581 raft_consensus.cc:2454] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482432aabf084ff5828eaa5f4915240a in term 0.
I20260812 00:37:34.756368 4705 leader_election.cc:302] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482432aabf084ff5828eaa5f4915240a, 858abe7d735d4151b42343a729fc3f9e; no voters:
I20260812 00:37:34.756767 5241 raft_consensus.cc:2781] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:37:34.756911 5241 raft_consensus.cc:482] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:37:34.757015 5241 raft_consensus.cc:3037] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:34.759413 5241 raft_consensus.cc:504] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.760040 5241 leader_election.cc:288] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [CANDIDATE]: Term 1 election: Requested vote from peers 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345), 858abe7d735d4151b42343a729fc3f9e (127.7.244.65:38521)
I20260812 00:37:34.760661 4581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a155f6b67b9d45c48977035f6ee75904" candidate_uuid: "482432aabf084ff5828eaa5f4915240a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858abe7d735d4151b42343a729fc3f9e"
I20260812 00:37:34.760663 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a155f6b67b9d45c48977035f6ee75904" candidate_uuid: "482432aabf084ff5828eaa5f4915240a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07194dc481514a97835962d7f46b7d92"
I20260812 00:37:34.760876 5116 raft_consensus.cc:3037] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:34.760879 4581 raft_consensus.cc:3037] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:34.763360 5116 raft_consensus.cc:2454] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482432aabf084ff5828eaa5f4915240a in term 1.
I20260812 00:37:34.763370 4581 raft_consensus.cc:2454] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482432aabf084ff5828eaa5f4915240a in term 1.
I20260812 00:37:34.763872 4705 leader_election.cc:302] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482432aabf084ff5828eaa5f4915240a, 858abe7d735d4151b42343a729fc3f9e; no voters:
I20260812 00:37:34.764225 5241 raft_consensus.cc:2781] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:37:34.764719 5241 raft_consensus.cc:686] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 1 LEADER]: Becoming Leader. State: Replica: 482432aabf084ff5828eaa5f4915240a, State: Running, Role: LEADER
I20260812 00:37:34.765317 5241 consensus_queue.cc:237] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } }
I20260812 00:37:34.770363 4271 catalog_manager.cc:5495] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a reported cstate change: term changed from 0 to 1, leader changed from <none> to 482432aabf084ff5828eaa5f4915240a (127.7.244.66). New cstate: current_term: 1 leader_uuid: "482432aabf084ff5828eaa5f4915240a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } health_report { overall_health: HEALTHY } } }
I20260812 00:37:34.817951 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.812040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51852 (local address 127.7.244.66:36563)
0812 00:37:34.812219 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.812224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.812275 (+ 51us) server_negotiation.cc:407] Connection header received
0812 00:37:34.812497 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.812501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.812599 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.812743 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:37:34.813143 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.814006 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.815575 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.816014 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.816119 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.816768 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:37:34.817380 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:37:34.817509 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.817622 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.817704 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 00:37:34.827145 4581 raft_consensus.cc:1261] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Refusing update from remote peer 482432aabf084ff5828eaa5f4915240a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:37:34.827240 5116 raft_consensus.cc:1261] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 1 FOLLOWER]: Refusing update from remote peer 482432aabf084ff5828eaa5f4915240a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:37:34.828303 5246 consensus_queue.cc:1035] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [LEADER]: Connected to new peer: Peer: permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:37:34.828738 5241 consensus_queue.cc:1035] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [LEADER]: Connected to new peer: Peer: permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:37:34.845142 5254 mvcc.cc:205] Tried to move back new op lower bound from 7317483744560496640 to 7317483744323133440. Current Snapshot: MvccSnapshot[applied={T|T < 7317483744560496640}]
I20260812 00:37:34.857609 4272 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:37:34.857936 4272 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:37:34.862239 4272 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Sending DeleteTablet for 3 replicas of tablet a155f6b67b9d45c48977035f6ee75904
I20260812 00:37:34.863759 4526 tablet_service.cc:1514] Processing DeleteTablet for tablet a155f6b67b9d45c48977035f6ee75904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:41584
I20260812 00:37:34.863770 5081 tablet_service.cc:1514] Processing DeleteTablet for tablet a155f6b67b9d45c48977035f6ee75904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:52494
I20260812 00:37:34.863835 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet a155f6b67b9d45c48977035f6ee75904 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:34 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:51850
I20260812 00:37:34.865406 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 4398
I20260812 00:37:34.873988 5271 tablet_replica.cc:330] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: stopping tablet replica
I20260812 00:37:34.874498 5271 raft_consensus.cc:2229] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:37:34.875013 5271 raft_consensus.cc:2258] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:37:34.877876 5271 ts_tablet_manager.cc:1905] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:37:34.879261 5273 tablet_replica.cc:330] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: stopping tablet replica
I20260812 00:37:34.879757 5273 raft_consensus.cc:2229] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:37:34.880363 5273 raft_consensus.cc:2258] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:37:34.881347 5275 tablet_replica.cc:330] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: stopping tablet replica
I20260812 00:37:34.881837 5275 raft_consensus.cc:2229] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:37:34.882624 5275 raft_consensus.cc:2258] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:37:34.885228 5273 ts_tablet_manager.cc:1905] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:37:34.887132 5271 ts_tablet_manager.cc:1918] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:37:34.887603 5271 log.cc:1196] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/a155f6b67b9d45c48977035f6ee75904
I20260812 00:37:34.888278 5271 ts_tablet_manager.cc:1939] T a155f6b67b9d45c48977035f6ee75904 P 07194dc481514a97835962d7f46b7d92: Deleting consensus metadata
I20260812 00:37:34.890475 4251 catalog_manager.cc:4841] TS 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345): tablet a155f6b67b9d45c48977035f6ee75904 (table test-table [id=50d30b5f283441e09340791e2bab4c0a]) successfully deleted
I20260812 00:37:34.890965 5275 ts_tablet_manager.cc:1905] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 00:37:34 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:37:34.898670 5275 ts_tablet_manager.cc:1918] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:37:34.898903 5275 log.cc:1196] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/a155f6b67b9d45c48977035f6ee75904
I20260812 00:37:34.899539 5275 ts_tablet_manager.cc:1939] T a155f6b67b9d45c48977035f6ee75904 P 482432aabf084ff5828eaa5f4915240a: Deleting consensus metadata
I20260812 00:37:34.900969 5273 ts_tablet_manager.cc:1918] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:37:34.901163 5273 log.cc:1196] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/a155f6b67b9d45c48977035f6ee75904
I20260812 00:37:34.901779 4245 catalog_manager.cc:4841] TS 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563): tablet a155f6b67b9d45c48977035f6ee75904 (table test-table [id=50d30b5f283441e09340791e2bab4c0a]) successfully deleted
I20260812 00:37:34.901893 5273 ts_tablet_manager.cc:1939] T a155f6b67b9d45c48977035f6ee75904 P 858abe7d735d4151b42343a729fc3f9e: Deleting consensus metadata
I20260812 00:37:34.903895 4246 catalog_manager.cc:4841] TS 858abe7d735d4151b42343a729fc3f9e (127.7.244.65:38521): tablet a155f6b67b9d45c48977035f6ee75904 (table test-table [id=50d30b5f283441e09340791e2bab4c0a]) successfully deleted
I20260812 00:37:34.904560 5223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:34.866772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41590 (local address 127.7.244.65:38521)
0812 00:37:34.866983 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:34.866990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:34.876069 (+ 9079us) server_negotiation.cc:407] Connection header received
0812 00:37:34.876377 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:34.876383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:34.876493 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:34.876606 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:34.883357 (+ 6751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.884611 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.887073 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:34.887794 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:34.887915 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:34.894403 (+ 6488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:34.894461 (+ 58us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:34.894486 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:34.894558 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:34.899996 (+ 5438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:34.901524 (+ 1528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:34.901536 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:34.901540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:34.901762 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:34.902498 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:34.902503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:34.902506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:34.903041 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:34.903419 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:34.903959 (+ 540us) server_negotiation.cc:299] Negotiation successful
0812 00:37:34.904221 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:37:34.905380 4659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:34.996201 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 4398
I20260812 00:37:35.021021 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 4679
Aug 12 00:37:35 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:37:35.044304 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:35.021984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.7.244.66:36563)
0812 00:37:35.022138 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:35.022142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:35.022161 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:37:35.022346 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:35.022350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:35.022419 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:35.022502 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:35.024238 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:35.025104 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:35.027278 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:35.027746 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:35.027848 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:35.037782 (+ 9934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:35.037799 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:35.037823 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:35.037867 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:35.041121 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:35.041985 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:35.041994 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:35.041996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:35.042146 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:35.042595 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:35.042599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:35.042601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:35.043018 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:35.043390 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:35.043685 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 00:37:35.043942 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 00:37:35.045130 4944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:35.159799 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 4679
I20260812 00:37:35.190955 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 4959
Aug 12 00:37:35 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:37:35.210228 5225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:35.191898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.7.244.67:36345)
0812 00:37:35.192122 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:35.192128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:35.192147 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:37:35.192210 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:35.192214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:35.192279 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:35.192383 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:35.193850 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:35.194609 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:35.195991 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:35.196421 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:35.196546 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:35.204672 (+ 8126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:35.204692 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:35.204710 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:35.204757 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:35.207588 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:35.208342 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:35.208350 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:35.208353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:35.208493 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:35.208821 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:35.208824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:35.208826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:35.209208 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:35.209474 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:35.209751 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 00:37:35.209945 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 00:37:35.210961 5191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:35.299961 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 4959
I20260812 00:37:35.323808 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 4217
I20260812 00:37:35.325028 4332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:35.440060 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 4217
I20260812 00:37:35.471707 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39087
--webserver_interface=127.7.244.126
--webserver_port=33239
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:39087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:35.847666 5288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:35.847998 5288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:35.848100 5288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:35.855702 5288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:35.855835 5288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:35.855882 5288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:35.855959 5288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:35.856005 5288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:35.866449 5288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:39087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=33239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:35.867841 5288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:35.869187 5288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:35.896492 5300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:35.896715 5299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:35.896449 5303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:35.898015 5288 server_base.cc:1034] running on GCE node
I20260812 00:37:35.899092 5288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:35.900899 5288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:35.902160 5288 hybrid_clock.cc:648] HybridClock initialized: now 1786495055902100 us; error 52 us; skew 500 ppm
I20260812 00:37:35.902637 5288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:35 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495055, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:35.906543 5288 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:37:35.908578 5288 webserver.cc:460] Webserver started at http://127.7.244.126:33239/ using document root <none> and password file <none>
I20260812 00:37:35.909497 5288 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:35.909667 5288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:35.917188 5288 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 00:37:35.920940 5320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:35.922772 5288 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:37:35.922998 5288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "3bf3cd2327f74025a7908a9a97c9fa4d"
format_stamp: "Formatted at 2026-08-12 00:37:32 on dist-test-slave-gjw7"
server_key: "c5f067dd214f90f68d7b609fe3949e9c"
server_key_iv: "968a32773a0b19c75cd8273877b9e8fc"
server_key_version: "encryptionkey@0"
I20260812 00:37:35.924248 5288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:35.942302 5288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:35.947322 5288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:35.947647 5288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:35.967983 5288 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:39087
I20260812 00:37:35.967994 5425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:39087 every 8 connection(s)
I20260812 00:37:35.969762 5288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:37:35.972663 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 5288
I20260812 00:37:35.973553 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:38521
--local_ip_for_outbound_sockets=127.7.244.65
--tserver_master_addrs=127.7.244.126:39087
--webserver_port=45283
--webserver_interface=127.7.244.65
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:37:35.977747 5426 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:37:35.981477 5426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Bootstrap starting.
I20260812 00:37:36.002475 5426 log.cc:824] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:36.016623 5426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:37:36.017223 5426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Bootstrap complete.
I20260812 00:37:36.026866 5426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:36.027878 5426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf3cd2327f74025a7908a9a97c9fa4d, State: Initialized, Role: FOLLOWER
I20260812 00:37:36.028661 5426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:36.028951 5426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:37:36.029086 5426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:37:36.029275 5426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:37:36.033947 5426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:36.034607 5426 leader_election.cc:302] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bf3cd2327f74025a7908a9a97c9fa4d; no voters:
I20260812 00:37:36.035295 5426 leader_election.cc:288] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 00:37:36.035454 5436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 2 FOLLOWER]: Leader election won for term 2
I20260812 00:37:36.037945 5426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:36.042670 5436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 2 LEADER]: Becoming Leader. State: Replica: 3bf3cd2327f74025a7908a9a97c9fa4d, State: Running, Role: LEADER
I20260812 00:37:36.043447 5436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:36.047106 5438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } } }
I20260812 00:37:36.047576 5438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:36.052358 5439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bf3cd2327f74025a7908a9a97c9fa4d. Latest consensus state: current_term: 2 leader_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } } }
I20260812 00:37:36.052748 5439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:36.053454 5455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:37:36.062315 5455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:37:36.064055 5455 catalog_manager.cc:1259] Loaded cluster ID: 9e0704bf31b1498ba26eb5a096db3ebb
I20260812 00:37:36.064252 5455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:37:36.067811 5455 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 00:37:36.085852 5455 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 00:37:36.096295 5455 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:37:36.098506 5455 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Loaded TSK: 0
I20260812 00:37:36.100541 5455 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:37:36.356208 5428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:36.356550 5428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:36.356676 5428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:36.364637 5428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:36.364771 5428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:36.364925 5428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:37:36.375531 5428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:38521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=45283
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:36.377039 5428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:36.378584 5428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:36.394570 5474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:36.394491 5475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:36.394847 5477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:36.395605 5428 server_base.cc:1034] running on GCE node
I20260812 00:37:36.396373 5428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:36.397634 5428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:36.398896 5428 hybrid_clock.cc:648] HybridClock initialized: now 1786495056398815 us; error 76 us; skew 500 ppm
I20260812 00:37:36.399291 5428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:36 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495056, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:36.403271 5428 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:37:36.405125 5428 webserver.cc:460] Webserver started at http://127.7.244.65:45283/ using document root <none> and password file <none>
I20260812 00:37:36.406008 5428 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:36.406183 5428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:36.412899 5428 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 00:37:36.416399 5488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:36.418300 5428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260812 00:37:36.418532 5428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "858abe7d735d4151b42343a729fc3f9e"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "db46d8b49b52b40e59d28053c71a3781"
server_key_iv: "68e8784f51c75408006eea9c2fa83d8a"
server_key_version: "encryptionkey@0"
I20260812 00:37:36.419761 5428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:36.457388 5428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:36.462579 5428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:36.462874 5428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:36.464042 5428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:36.465795 5428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:36.465919 5428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:36.466039 5428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:36.466128 5428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:36.499634 5428 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:38521
I20260812 00:37:36.499701 5708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:38521 every 8 connection(s)
I20260812 00:37:36.501264 5428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:37:36.503911 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 5428
I20260812 00:37:36.504779 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:36563
--local_ip_for_outbound_sockets=127.7.244.66
--tserver_master_addrs=127.7.244.126:39087
--webserver_port=45335
--webserver_interface=127.7.244.66
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:37:36 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495056, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:36.529364 5716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:36.504795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:39899 (local address 127.7.244.126:39087)
0812 00:37:36.509749 (+ 4954us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:36.509763 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:36.509807 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:37:36.510138 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:36.510152 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:36.510536 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:36.510929 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:36.512722 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:36.514076 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:36.515770 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:36.516473 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:36.516664 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:36.520783 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:36.520792 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:36.520810 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:36.520855 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:36.524053 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:36.525123 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:36.525132 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:36.525138 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:36.525397 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:36.525924 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:36.525929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:36.525931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:36.526490 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:36.526683 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:36.527579 (+ 896us) server_negotiation.cc:299] Negotiation successful
0812 00:37:36.528046 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4728,"thread_start_us":4603,"threads_started":1}
I20260812 00:37:36.529361 5719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:36.505140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:39899)
0812 00:37:36.506133 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:36.506185 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:36.508030 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:36.511152 (+ 3122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:36.511165 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:36.511856 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:36.512453 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:36.512474 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:36.514284 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:36.514289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:36.515594 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:36.515610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:36.516754 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:36.516764 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:36.517047 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:36.517812 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:36.517842 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:36.520427 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:36.524244 (+ 3817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:36.524256 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:36.524271 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:36.524941 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:36.525542 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:36.525548 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:36.525552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:36.525786 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:36.526660 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:36.526669 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:36.527034 (+ 365us) client_negotiation.cc:769] Sending connection context
0812 00:37:36.527299 (+ 265us) client_negotiation.cc:240] Negotiation successful
0812 00:37:36.527939 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":415,"threads_started":1}
I20260812 00:37:36.532636 5711 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39087
I20260812 00:37:36.533049 5711 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:36.534123 5711 heartbeater.cc:511] Master 127.7.244.126:39087 requested a full tablet report, sending...
I20260812 00:37:36.536897 5351 ts_manager.cc:194] Registered new tserver with Master: 858abe7d735d4151b42343a729fc3f9e (127.7.244.65:38521)
I20260812 00:37:36.538622 5351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:39899
W20260812 00:37:36.866357 5721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:36.866675 5721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:36.866802 5721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:36.874483 5721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:36.874629 5721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:36.874780 5721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:37:36.885192 5721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:36563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=45335
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:36.886623 5721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:36.888006 5721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:36.905331 5737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:36.905357 5739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:36.905777 5736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:36.906699 5721 server_base.cc:1034] running on GCE node
I20260812 00:37:36.907537 5721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:36.908910 5721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:36.910264 5721 hybrid_clock.cc:648] HybridClock initialized: now 1786495056910166 us; error 91 us; skew 500 ppm
I20260812 00:37:36.910691 5721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:36 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495056, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:36.916424 5721 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:37:36.918576 5721 webserver.cc:460] Webserver started at http://127.7.244.66:45335/ using document root <none> and password file <none>
I20260812 00:37:36.919382 5721 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:36.919525 5721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:36.931335 5721 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.004s
I20260812 00:37:36.935084 5758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:36.937101 5721 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 00:37:36.937357 5721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "482432aabf084ff5828eaa5f4915240a"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "1677ef2d9f1b24495347bbd920bddf2b"
server_key_iv: "c0966c940467c4cf0adfff2a297bb747"
server_key_version: "encryptionkey@0"
I20260812 00:37:36.938889 5721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:36.967006 5721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:36.972193 5721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:36.972517 5721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:36.973721 5721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:36.975241 5721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:36.975347 5721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:36.975462 5721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:36.975533 5721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:37.010406 5721 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:36563
I20260812 00:37:37.010536 5953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:36563 every 8 connection(s)
I20260812 00:37:37.012238 5721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:37:37.018559 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 5721
I20260812 00:37:37.019220 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:36345
--local_ip_for_outbound_sockets=127.7.244.67
--tserver_master_addrs=127.7.244.126:39087
--webserver_port=33919
--webserver_interface=127.7.244.67
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:37:37 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495056, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:37.035702 5716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.015441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:60507 (local address 127.7.244.126:39087)
0812 00:37:37.015677 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.015682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.017095 (+ 1413us) server_negotiation.cc:407] Connection header received
0812 00:37:37.019345 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.019352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.019473 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.019569 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:37.021487 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.022214 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.024019 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.024477 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.024595 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.028699 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:37.028705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:37.028708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:37.028751 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:37.031729 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.032684 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.032689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.032691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.032889 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.033645 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.033650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.033652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.033974 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:37.034118 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.035184 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.035443 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 00:37:37.037241 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.016009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:60507)
0812 00:37:37.016874 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:37.016929 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:37.018971 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:37.019896 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:37.019920 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:37.020507 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:37.021184 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.021208 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.022409 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.022415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.023807 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.023823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.024738 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.024747 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.025023 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.025747 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:37.025772 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:37.028411 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:37.031945 (+ 3534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:37.031957 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:37.031975 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:37.032518 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:37.033092 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:37.033097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:37.033101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:37.033462 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:37.034148 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:37.034158 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:37.034676 (+ 518us) client_negotiation.cc:769] Sending connection context
0812 00:37:37.035022 (+ 346us) client_negotiation.cc:240] Negotiation successful
0812 00:37:37.035663 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":354,"threads_started":1}
I20260812 00:37:37.039408 5954 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39087
I20260812 00:37:37.039819 5954 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:37.040861 5954 heartbeater.cc:511] Master 127.7.244.126:39087 requested a full tablet report, sending...
I20260812 00:37:37.042989 5351 ts_manager.cc:194] Registered new tserver with Master: 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563)
I20260812 00:37:37.044255 5351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:60507
W20260812 00:37:37.380506 5962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:37.380824 5962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:37.380925 5962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:37.388387 5962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:37.388698 5962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:37.389032 5962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:37:37.400677 5962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:36345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=33919
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:37.402180 5962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:37.403788 5962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:37.418601 5969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:37.418548 5979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:37.418578 5971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:37.419562 5962 server_base.cc:1034] running on GCE node
I20260812 00:37:37.420258 5962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:37.421336 5962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:37.422607 5962 hybrid_clock.cc:648] HybridClock initialized: now 1786495057422538 us; error 65 us; skew 500 ppm
I20260812 00:37:37.422979 5962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:37 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495057, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:37.426542 5962 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:37:37.428153 5962 webserver.cc:460] Webserver started at http://127.7.244.67:33919/ using document root <none> and password file <none>
I20260812 00:37:37.428944 5962 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:37.429083 5962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:37.435688 5962 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:37:37.439157 5994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:37.440833 5962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 00:37:37.441012 5962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "07194dc481514a97835962d7f46b7d92"
format_stamp: "Formatted at 2026-08-12 00:37:34 on dist-test-slave-gjw7"
server_key: "0600622515e311130d87b31976d809a1"
server_key_iv: "4a9415f965500195c465aaab0c24bb27"
server_key_version: "encryptionkey@0"
I20260812 00:37:37.442154 5962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:37.458655 5962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:37.463631 5962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:37.463905 5962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:37.464982 5962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:37.466601 5962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:37.466701 5962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:37.466818 5962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:37.466894 5962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:37.497694 5962 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:36345
I20260812 00:37:37.497779 6206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:36345 every 8 connection(s)
I20260812 00:37:37.499326 5962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:37:37.500922 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 5962
Aug 12 00:37:37 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495057, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:37.518970 5716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.502828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44278 (local address 127.7.244.126:39087)
0812 00:37:37.503237 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.503243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.503264 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:37:37.503333 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.503339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.503427 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.503538 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:37.505708 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.506822 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.508359 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.508837 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.508991 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.512155 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:37.512161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:37.512164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:37.512212 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:37.515509 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.516459 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.516466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.516469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.516662 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.517153 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.517160 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.517162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.517549 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:37.517910 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.518258 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.518569 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294}
I20260812 00:37:37.523422 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.502993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:36913 (local address 127.7.244.126:39087)
0812 00:37:37.506618 (+ 3625us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.506626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.506651 (+ 25us) server_negotiation.cc:407] Connection header received
0812 00:37:37.506962 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.506968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.507055 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.507150 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:37.508927 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.509895 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.511659 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.512123 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.512258 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.516570 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:37.516575 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:37.516578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:37.516617 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:37.519679 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.520728 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.520733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.520736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.520922 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.521606 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.521610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.521612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.521940 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:37.522064 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.522880 (+ 816us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.523080 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3490,"thread_start_us":3713,"threads_started":1}
I20260812 00:37:37.524422 6214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.502797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:36913)
0812 00:37:37.504501 (+ 1704us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:37.504558 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:37.506625 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:37.507487 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:37.507498 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:37.508076 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:37.508709 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.508728 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.510078 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.510082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.511475 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.511492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.512390 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.512399 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.512669 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.513406 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:37.513433 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:37.516284 (+ 2851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:37.519873 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:37.519887 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:37.519908 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:37.520584 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:37.521077 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:37.521084 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:37.521087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:37.521485 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:37.522094 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:37.522104 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:37.522518 (+ 414us) client_negotiation.cc:769] Sending connection context
0812 00:37:37.522777 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 00:37:37.523253 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":348,"threads_started":1}
I20260812 00:37:37.525914 6208 heartbeater.cc:347] Connected to a master server at 127.7.244.126:39087
I20260812 00:37:37.526221 6208 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:37.526985 6208 heartbeater.cc:511] Master 127.7.244.126:39087 requested a full tablet report, sending...
I20260812 00:37:37.528537 5351 ts_manager.cc:194] Registered new tserver with Master: 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345)
I20260812 00:37:37.529495 5351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:36913
I20260812 00:37:37.535013 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:37:37.543044 5711 heartbeater.cc:503] Master 127.7.244.126:39087 was elected leader, sending a full tablet report...
I20260812 00:37:37.543272 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.536143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44282 (local address 127.7.244.126:39087)
0812 00:37:37.536370 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.536377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.536763 (+ 386us) server_negotiation.cc:407] Connection header received
0812 00:37:37.537040 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.537045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.537255 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.537455 (+ 200us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:37:37.538122 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.538964 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.540659 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.541083 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.541179 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.541880 (+ 701us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:37:37.542577 (+ 697us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:37:37.542721 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.542894 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.542997 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:37:37.544461 5351 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:37:37.552227 5351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:37:37.578014 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.565698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.7.244.66:36563)
0812 00:37:37.567504 (+ 1806us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.567514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.567545 (+ 31us) server_negotiation.cc:407] Connection header received
0812 00:37:37.567649 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.567659 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.567944 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.568156 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:37.569801 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.571784 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.575556 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.576631 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.576787 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.577051 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.577432 (+ 381us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.577626 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1635,"thread_start_us":1514,"threads_started":1}
I20260812 00:37:37.578075 6226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.565502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:36563 (local address 127.0.0.1:51862)
0812 00:37:37.566779 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:37.566839 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:37.567031 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:37.568196 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:37.568204 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:37.568236 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:37.569498 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.569519 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.571945 (+ 2426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.571951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.575283 (+ 3332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.575297 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.576885 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.576892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.577122 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.577171 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 00:37:37.577332 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 00:37:37.577626 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":157,"thread_start_us":830,"threads_started":1}
I20260812 00:37:37.578384 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.568082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41600)
0812 00:37:37.570360 (+ 2278us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:37.570379 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:37.570499 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:37.571446 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:37.571451 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:37.571477 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:37.571844 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.571856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.573236 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.573241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.575363 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.575379 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.577558 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.577565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.577723 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.577750 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 00:37:37.577806 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 00:37:37.577970 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2113,"mutex_wait_us":95,"thread_start_us":1956,"threads_started":1}
I20260812 00:37:37.578469 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.566469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.7.244.67:36345)
0812 00:37:37.567663 (+ 1194us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.567672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.568334 (+ 662us) server_negotiation.cc:407] Connection header received
0812 00:37:37.568454 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.568461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.568697 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.568913 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:37.570504 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.571863 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.575076 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.576795 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.576951 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.577198 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.577933 (+ 735us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.578115 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1024,"thread_start_us":977,"threads_started":1}
I20260812 00:37:37.579975 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.565748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:36345 (local address 127.0.0.1:52512)
0812 00:37:37.568054 (+ 2306us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:37.568072 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:37.568197 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:37.569230 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:37.569237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:37.569267 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:37.570275 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.570290 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.572005 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.572010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.574891 (+ 2881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.574910 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.577430 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.577438 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.577657 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.577696 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 00:37:37.577793 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 00:37:37.579590 (+ 1797us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2177,"mutex_wait_us":1808,"thread_start_us":2094,"threads_started":1}
I20260812 00:37:37.580953 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.568797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.7.244.65:38521)
0812 00:37:37.569882 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.569893 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.570626 (+ 733us) server_negotiation.cc:407] Connection header received
0812 00:37:37.570734 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.570742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.571238 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.571439 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:37.572066 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.573075 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.575579 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.577275 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.580266 (+ 2991us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.580490 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.580561 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.580703 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":706,"threads_started":1}
I20260812 00:37:37.584017 5585 tablet_service.cc:1467] Processing CreateTablet for tablet 408ba4ca6a6549b19896b0ac57bd9e4d (DEFAULT_TABLE table=test-table [id=ec8f8afebb41476b9ac3c495c8fd74bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:37.584115 6083 tablet_service.cc:1467] Processing CreateTablet for tablet 408ba4ca6a6549b19896b0ac57bd9e4d (DEFAULT_TABLE table=test-table [id=ec8f8afebb41476b9ac3c495c8fd74bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:37.583455 5846 tablet_service.cc:1467] Processing CreateTablet for tablet 408ba4ca6a6549b19896b0ac57bd9e4d (DEFAULT_TABLE table=test-table [id=ec8f8afebb41476b9ac3c495c8fd74bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:37.585706 5585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408ba4ca6a6549b19896b0ac57bd9e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:37.585716 6083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408ba4ca6a6549b19896b0ac57bd9e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:37.585711 5846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 408ba4ca6a6549b19896b0ac57bd9e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:37.600574 6240 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Bootstrap starting.
I20260812 00:37:37.603143 6243 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: Bootstrap starting.
I20260812 00:37:37.604791 6240 tablet_bootstrap.cc:654] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:37.606331 6240 log.cc:824] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:37.607309 6243 tablet_bootstrap.cc:654] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:37.608114 6242 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Bootstrap starting.
I20260812 00:37:37.608955 6243 log.cc:824] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:37.611661 6240 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: No bootstrap required, opened a new log
I20260812 00:37:37.612042 6240 ts_tablet_manager.cc:1397] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.004s
I20260812 00:37:37.612093 6242 tablet_bootstrap.cc:654] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:37.612673 6243 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: No bootstrap required, opened a new log
I20260812 00:37:37.612972 6243 ts_tablet_manager.cc:1397] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 00:37:37.613626 6242 log.cc:824] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:37.617395 6242 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: No bootstrap required, opened a new log
I20260812 00:37:37.617715 6242 ts_tablet_manager.cc:1397] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260812 00:37:37.620842 6240 raft_consensus.cc:348] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.621311 6240 raft_consensus.cc:374] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:37.621449 6240 raft_consensus.cc:729] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858abe7d735d4151b42343a729fc3f9e, State: Initialized, Role: FOLLOWER
I20260812 00:37:37.622115 6240 consensus_queue.cc:260] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.623867 6240 ts_tablet_manager.cc:1428] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260812 00:37:37.623548 6243 raft_consensus.cc:348] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.624038 6243 raft_consensus.cc:374] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:37.624148 6243 raft_consensus.cc:729] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482432aabf084ff5828eaa5f4915240a, State: Initialized, Role: FOLLOWER
I20260812 00:37:37.624850 6243 consensus_queue.cc:260] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.631224 6242 raft_consensus.cc:348] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.631794 6242 raft_consensus.cc:374] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:37.631919 6242 raft_consensus.cc:729] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07194dc481514a97835962d7f46b7d92, State: Initialized, Role: FOLLOWER
I20260812 00:37:37.632252 5954 heartbeater.cc:503] Master 127.7.244.126:39087 was elected leader, sending a full tablet report...
I20260812 00:37:37.632779 6242 consensus_queue.cc:260] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.633481 6243 ts_tablet_manager.cc:1428] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: Time spent starting tablet: real 0.020s user 0.013s sys 0.008s
I20260812 00:37:37.636994 6208 heartbeater.cc:503] Master 127.7.244.126:39087 was elected leader, sending a full tablet report...
I20260812 00:37:37.637629 6242 ts_tablet_manager.cc:1428] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Time spent starting tablet: real 0.020s user 0.014s sys 0.004s
I20260812 00:37:37.665515 6250 raft_consensus.cc:482] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:37:37.665906 6250 raft_consensus.cc:504] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.667690 6250 leader_election.cc:288] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345), 858abe7d735d4151b42343a729fc3f9e (127.7.244.65:38521)
I20260812 00:37:37.675339 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.668520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:49203 (local address 127.7.244.65:38521)
0812 00:37:37.668841 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.668847 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.669333 (+ 486us) server_negotiation.cc:407] Connection header received
0812 00:37:37.669566 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.669572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.669651 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.669736 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:37.670333 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.671177 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.673226 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.674222 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.674361 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.674421 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.674890 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.675071 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260812 00:37:37.675423 6252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.668008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.7.244.66:49203)
0812 00:37:37.669177 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:37.669210 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:37.669401 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:37.669766 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:37.669771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:37.669797 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:37.670181 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.670192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.671327 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.671333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.673017 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.673032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.674476 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.674484 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.674658 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.674708 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 00:37:37.674807 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:37:37.675006 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1042,"mutex_wait_us":1,"thread_start_us":993,"threads_started":1}
I20260812 00:37:37.675753 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.668162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:38345 (local address 127.7.244.67:36345)
0812 00:37:37.668654 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.668660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.669527 (+ 867us) server_negotiation.cc:407] Connection header received
0812 00:37:37.669792 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.669798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.669873 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.669929 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:37.670725 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.671667 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.673668 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.674652 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.674835 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.674907 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.675384 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.675528 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329}
I20260812 00:37:37.675997 6253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.668145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:36345 (local address 127.7.244.66:38345)
0812 00:37:37.669428 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:37.669443 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:37.669629 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:37.670112 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:37.670116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:37.670146 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:37.670523 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.670535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.671846 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.671850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.673479 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:37.673495 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.674949 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.674956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:37.675171 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.675208 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 00:37:37.675334 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 00:37:37.675484 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"thread_start_us":702,"threads_started":1}
I20260812 00:37:37.676733 5620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408ba4ca6a6549b19896b0ac57bd9e4d" candidate_uuid: "482432aabf084ff5828eaa5f4915240a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858abe7d735d4151b42343a729fc3f9e" is_pre_election: true
I20260812 00:37:37.676949 6118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408ba4ca6a6549b19896b0ac57bd9e4d" candidate_uuid: "482432aabf084ff5828eaa5f4915240a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07194dc481514a97835962d7f46b7d92" is_pre_election: true
I20260812 00:37:37.677284 5620 raft_consensus.cc:2454] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482432aabf084ff5828eaa5f4915240a in term 0.
I20260812 00:37:37.677456 6118 raft_consensus.cc:2454] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482432aabf084ff5828eaa5f4915240a in term 0.
I20260812 00:37:37.678081 5766 leader_election.cc:302] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482432aabf084ff5828eaa5f4915240a, 858abe7d735d4151b42343a729fc3f9e; no voters:
I20260812 00:37:37.678550 6250 raft_consensus.cc:2781] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:37:37.678694 6250 raft_consensus.cc:482] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:37:37.678802 6250 raft_consensus.cc:3037] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:37.681267 6250 raft_consensus.cc:504] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.681983 6250 leader_election.cc:288] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [CANDIDATE]: Term 1 election: Requested vote from peers 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345), 858abe7d735d4151b42343a729fc3f9e (127.7.244.65:38521)
I20260812 00:37:37.682619 5620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408ba4ca6a6549b19896b0ac57bd9e4d" candidate_uuid: "482432aabf084ff5828eaa5f4915240a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858abe7d735d4151b42343a729fc3f9e"
I20260812 00:37:37.682616 6118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "408ba4ca6a6549b19896b0ac57bd9e4d" candidate_uuid: "482432aabf084ff5828eaa5f4915240a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07194dc481514a97835962d7f46b7d92"
I20260812 00:37:37.682833 5620 raft_consensus.cc:3037] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:37.682840 6118 raft_consensus.cc:3037] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:37.685209 6118 raft_consensus.cc:2454] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482432aabf084ff5828eaa5f4915240a in term 1.
I20260812 00:37:37.685218 5620 raft_consensus.cc:2454] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482432aabf084ff5828eaa5f4915240a in term 1.
I20260812 00:37:37.685788 5766 leader_election.cc:302] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482432aabf084ff5828eaa5f4915240a, 858abe7d735d4151b42343a729fc3f9e; no voters:
I20260812 00:37:37.686169 6250 raft_consensus.cc:2781] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:37:37.686700 6250 raft_consensus.cc:686] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 1 LEADER]: Becoming Leader. State: Replica: 482432aabf084ff5828eaa5f4915240a, State: Running, Role: LEADER
I20260812 00:37:37.687299 6250 consensus_queue.cc:237] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:37.692492 5351 catalog_manager.cc:5495] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a reported cstate change: term changed from 0 to 1, leader changed from <none> to 482432aabf084ff5828eaa5f4915240a (127.7.244.66). New cstate: current_term: 1 leader_uuid: "482432aabf084ff5828eaa5f4915240a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } health_report { overall_health: UNKNOWN } } }
I20260812 00:37:37.738294 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.732417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.7.244.66:36563)
0812 00:37:37.732659 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.732665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.732686 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:37:37.732821 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.732825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.732920 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.733065 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:37:37.733553 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.734292 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.735870 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.736296 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.736407 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.737015 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:37:37.737627 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:37:37.737769 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.737895 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.737996 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:37:37.747507 6118 raft_consensus.cc:1261] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 1 FOLLOWER]: Refusing update from remote peer 482432aabf084ff5828eaa5f4915240a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:37:37.747553 5620 raft_consensus.cc:1261] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Refusing update from remote peer 482432aabf084ff5828eaa5f4915240a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:37:37.748600 6254 consensus_queue.cc:1035] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [LEADER]: Connected to new peer: Peer: permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:37:37.748960 6250 consensus_queue.cc:1035] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [LEADER]: Connected to new peer: Peer: permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 00:37:37.752051 6212 tablet.cc:2369] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:37:37.754585 5713 tablet.cc:2369] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:37:37.779341 5351 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:37:37.779668 5351 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:37:37.783643 5351 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Sending DeleteTablet for 3 replicas of tablet 408ba4ca6a6549b19896b0ac57bd9e4d
I20260812 00:37:37.784950 5585 tablet_service.cc:1514] Processing DeleteTablet for tablet 408ba4ca6a6549b19896b0ac57bd9e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:37 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:41600
I20260812 00:37:37.784972 5846 tablet_service.cc:1514] Processing DeleteTablet for tablet 408ba4ca6a6549b19896b0ac57bd9e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:37 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:51862
I20260812 00:37:37.785010 6083 tablet_service.cc:1514] Processing DeleteTablet for tablet 408ba4ca6a6549b19896b0ac57bd9e4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:37:37 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:52512
I20260812 00:37:37.786437 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 5428
I20260812 00:37:37.800560 6284 tablet_replica.cc:330] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: stopping tablet replica
I20260812 00:37:37.801252 6283 tablet_replica.cc:330] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: stopping tablet replica
I20260812 00:37:37.801890 6283 raft_consensus.cc:2229] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:37:37.801882 6284 raft_consensus.cc:2229] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:37:37.802597 6283 raft_consensus.cc:2258] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:37:37.802650 6284 raft_consensus.cc:2258] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:37:37.805018 6284 ts_tablet_manager.cc:1905] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:37:37.808593 6283 ts_tablet_manager.cc:1905] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:37:37.810352 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.787446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.7.244.65:38521)
0812 00:37:37.787612 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.787617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.787688 (+ 71us) server_negotiation.cc:407] Connection header received
0812 00:37:37.787890 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.787895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.787974 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.788083 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:37.790135 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.791362 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.793380 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.794235 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.794376 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.800474 (+ 6098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:37.800501 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:37.800525 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:37.800583 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:37.804917 (+ 4334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.806204 (+ 1287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.806217 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.806221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.806435 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.807015 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.807021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.807024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.807607 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:37.807964 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.809563 (+ 1599us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.809863 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 00:37:37.811528 5701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:37.814322 6284 ts_tablet_manager.cc:1918] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:37:37.814654 6284 log.cc:1196] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/408ba4ca6a6549b19896b0ac57bd9e4d
I20260812 00:37:37.815464 6284 ts_tablet_manager.cc:1939] T 408ba4ca6a6549b19896b0ac57bd9e4d P 482432aabf084ff5828eaa5f4915240a: Deleting consensus metadata
I20260812 00:37:37.818099 5321 catalog_manager.cc:4841] TS 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563): tablet 408ba4ca6a6549b19896b0ac57bd9e4d (table test-table [id=ec8f8afebb41476b9ac3c495c8fd74bf]) successfully deleted
I20260812 00:37:37.820665 6283 ts_tablet_manager.cc:1918] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:37:37.820907 6283 log.cc:1196] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/408ba4ca6a6549b19896b0ac57bd9e4d
I20260812 00:37:37.821724 6283 ts_tablet_manager.cc:1939] T 408ba4ca6a6549b19896b0ac57bd9e4d P 07194dc481514a97835962d7f46b7d92: Deleting consensus metadata
I20260812 00:37:37.824982 5324 catalog_manager.cc:4841] TS 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345): tablet 408ba4ca6a6549b19896b0ac57bd9e4d (table test-table [id=ec8f8afebb41476b9ac3c495c8fd74bf]) successfully deleted
I20260812 00:37:37.917361 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 5428
W20260812 00:37:37.961447 5322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.244.65:38521 (error 108)
I20260812 00:37:37.961982 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 5721
I20260812 00:37:37.963889 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.963330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41620)
0812 00:37:37.963550 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:37.963688 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260812 00:37:37.964164 5322 catalog_manager.cc:4584] TS 858abe7d735d4151b42343a729fc3f9e (127.7.244.65:38521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 408ba4ca6a6549b19896b0ac57bd9e4d: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
I20260812 00:37:37.982807 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:37.962970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51886 (local address 127.7.244.66:36563)
0812 00:37:37.963168 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:37.963175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:37.963198 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:37:37.963446 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:37.963451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:37.963551 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:37.963669 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:37.966074 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.967429 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.969691 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:37.970452 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:37.970616 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:37.974520 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:37.974549 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:37.974573 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:37.974638 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:37.979061 (+ 4423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.980175 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.980187 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.980191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.980415 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:37.980875 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:37.980880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:37.980883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:37.981456 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:37.981807 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:37.982144 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 00:37:37.982407 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:37:37.983682 5948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:38.018095 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:38.017506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41634)
0812 00:37:38.017756 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:38.017887 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 00:37:38.019328 6232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:38.018884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41646)
0812 00:37:38.019040 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:38.019159 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 00:37:38.080017 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:38.079533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41654)
0812 00:37:38.079758 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:38.079857 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 00:37:38.081164 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:38.080814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41668)
0812 00:37:38.080958 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:38.081045 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 00:37:38.105706 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 5721
I20260812 00:37:38.136929 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 5962
I20260812 00:37:38.151682 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:38.137733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.7.244.67:36345)
0812 00:37:38.137959 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:38.137965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:38.137990 (+ 25us) server_negotiation.cc:407] Connection header received
0812 00:37:38.138140 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:38.138175 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:38.138278 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:38.138394 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:38.139897 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:38.140636 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:38.142124 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:38.142559 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:38.142678 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:38.145521 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:38.145541 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:38.145559 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:38.145618 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:38.148721 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:38.149645 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:38.149654 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:38.149657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:38.149827 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:38.150185 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:38.150189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:38.150192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:38.150603 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:38.150835 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:38.151137 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 00:37:38.151324 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 00:37:38.152328 6195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:38.162060 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:38.161530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41676)
0812 00:37:38.161781 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:38.161908 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 00:37:38.163252 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:38.162771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41680)
0812 00:37:38.162981 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:38.163058 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260812 00:37:38.242246 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 5962
I20260812 00:37:38.273582 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 5288
I20260812 00:37:38.274914 5410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:38.391705 6237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:38.391139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38521 (local address 127.0.0.1:41684)
0812 00:37:38.391346 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:38.391497 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:38521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 00:37:38.391858 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 5288
I20260812 00:37:38.426882 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39087
--webserver_interface=127.7.244.126
--webserver_port=33239
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:39087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:38.794337 6296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:38.794713 6296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:38.794859 6296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:38.803049 6296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:38.803197 6296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:38.803252 6296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:38.803339 6296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:38.803412 6296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:38.814082 6296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:39087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:39087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=33239
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:38.815629 6296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:38.817102 6296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:38.829792 6303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:38.829764 6304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:38.829823 6307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:38.830616 6296 server_base.cc:1034] running on GCE node
I20260812 00:37:38.831511 6296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:38.833119 6296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:38.834360 6296 hybrid_clock.cc:648] HybridClock initialized: now 1786495058834302 us; error 76 us; skew 500 ppm
I20260812 00:37:38.834755 6296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:38 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495058, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:38.838330 6296 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:37:38.839982 6296 webserver.cc:460] Webserver started at http://127.7.244.126:33239/ using document root <none> and password file <none>
I20260812 00:37:38.840749 6296 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:38.840881 6296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:38.846978 6296 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 00:37:38.852867 6317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:38.854516 6296 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 00:37:38.854730 6296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "3bf3cd2327f74025a7908a9a97c9fa4d"
format_stamp: "Formatted at 2026-08-12 00:37:32 on dist-test-slave-gjw7"
server_key: "c5f067dd214f90f68d7b609fe3949e9c"
server_key_iv: "968a32773a0b19c75cd8273877b9e8fc"
server_key_version: "encryptionkey@0"
I20260812 00:37:38.855913 6296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:38.871145 6296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:38.875656 6296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:38.875976 6296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:38.908953 6296 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:39087
I20260812 00:37:38.908963 6396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:39087 every 8 connection(s)
I20260812 00:37:38.911290 6296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:37:38.917382 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 6296
I20260812 00:37:38.918272 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:38521
--local_ip_for_outbound_sockets=127.7.244.65
--tserver_master_addrs=127.7.244.126:39087
--webserver_port=45283
--webserver_interface=127.7.244.65
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:37:38.922178 6397 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:37:38.925885 6397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Bootstrap starting.
I20260812 00:37:38.947708 6397 log.cc:824] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:38.965888 6397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:37:38.966499 6397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d: Bootstrap complete.
I20260812 00:37:38.977270 6397 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:38.978368 6397 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf3cd2327f74025a7908a9a97c9fa4d, State: Initialized, Role: FOLLOWER
I20260812 00:37:38.979063 6397 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:38.979331 6397 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:37:38.979455 6397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:37:38.979629 6397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 2 FOLLOWER]: Advancing to term 3
I20260812 00:37:38.983906 6397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:38.984483 6397 leader_election.cc:302] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bf3cd2327f74025a7908a9a97c9fa4d; no voters:
I20260812 00:37:38.985241 6397 leader_election.cc:288] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 00:37:38.985417 6402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 3 FOLLOWER]: Leader election won for term 3
I20260812 00:37:38.986601 6402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [term 3 LEADER]: Becoming Leader. State: Replica: 3bf3cd2327f74025a7908a9a97c9fa4d, State: Running, Role: LEADER
I20260812 00:37:38.987262 6402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } }
I20260812 00:37:38.987695 6397 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:38.990125 6411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bf3cd2327f74025a7908a9a97c9fa4d. Latest consensus state: current_term: 3 leader_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } } }
I20260812 00:37:38.990262 6407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bf3cd2327f74025a7908a9a97c9fa4d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 39087 } } }
I20260812 00:37:38.990557 6411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:38.990623 6407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bf3cd2327f74025a7908a9a97c9fa4d [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:39.002255 6421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:37:39.012355 6421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:37:39.014138 6421 catalog_manager.cc:1259] Loaded cluster ID: 9e0704bf31b1498ba26eb5a096db3ebb
I20260812 00:37:39.014297 6421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 00:37:39.016821 6421 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f63571a39ad google::LogMessage::Fail() at ??:0
@ 0x7f63571a688a google::LogMessage::SendToLog() at ??:0
@ 0x7f63571a34de google::LogMessage::Flush() at ??:0
@ 0x7f63571a4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f636cbda512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f636cbd7976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f636cc5f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f636cc5f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6359def345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6359df291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6359df2799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6359dcdeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f635d0a76db start_thread at ??:0
@ 0x7f635603d61f clone at ??:0
W20260812 00:37:39.348631 6400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:39.348951 6400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:39.349071 6400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:39.356978 6400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:39.357141 6400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:39.357344 6400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:37:39.368815 6400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:38521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=45283
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:39.370298 6400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:39.371747 6400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:39.385864 6465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:39.385987 6468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:39.385950 6466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:39.386911 6400 server_base.cc:1034] running on GCE node
I20260812 00:37:39.387638 6400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:39.388691 6400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:39.389941 6400 hybrid_clock.cc:648] HybridClock initialized: now 1786495059389880 us; error 51 us; skew 500 ppm
I20260812 00:37:39.390312 6400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:39 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495059, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:39.393839 6400 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:37:39.395605 6400 webserver.cc:460] Webserver started at http://127.7.244.65:45283/ using document root <none> and password file <none>
I20260812 00:37:39.396456 6400 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:39.396600 6400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:39.403219 6400 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 00:37:39.406697 6485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:39.408437 6400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:37:39.408641 6400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "858abe7d735d4151b42343a729fc3f9e"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "db46d8b49b52b40e59d28053c71a3781"
server_key_iv: "68e8784f51c75408006eea9c2fa83d8a"
server_key_version: "encryptionkey@0"
I20260812 00:37:39.409828 6400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:39.452122 6400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:39.457017 6400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:39.457367 6400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:39.458477 6400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:39.460630 6496 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 00:37:39.466990 6400 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 00:37:39.467164 6400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260812 00:37:39.467327 6400 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 00:37:39.471237 6400 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 00:37:39.471359 6400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20260812 00:37:39.471817 6496 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Bootstrap starting.
I20260812 00:37:39.500837 6496 log.cc:824] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:39.509905 6400 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:38521
I20260812 00:37:39.509951 6704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:38521 every 8 connection(s)
I20260812 00:37:39.511577 6400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:37:39.512581 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 6400
I20260812 00:37:39.513530 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:36563
--local_ip_for_outbound_sockets=127.7.244.66
--tserver_master_addrs=127.7.244.126:39087
--webserver_port=45335
--webserver_interface=127.7.244.66
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:37:39.518087 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:39.515292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:58855)
0812 00:37:39.516300 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:39.516633 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":391,"threads_started":1}
I20260812 00:37:39.518306 6496 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:37:39.519100 6496 tablet_bootstrap.cc:492] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Bootstrap complete.
W20260812 00:37:39.519475 6707 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:39087 (0 consecutive failures): Network error: Failed to ping master at 127.7.244.126:39087: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
I20260812 00:37:39.520896 6496 ts_tablet_manager.cc:1397] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Time spent bootstrapping tablet: real 0.049s user 0.025s sys 0.018s
I20260812 00:37:39.526419 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:39.520220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:39499)
0812 00:37:39.526041 (+ 5821us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:39.526188 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5687,"threads_started":1}
I20260812 00:37:39.528873 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:39.528135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:54525)
0812 00:37:39.528342 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:39.528605 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 00:37:39.529843 6707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 00:37:39.535542 6496 raft_consensus.cc:348] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:39.536628 6496 raft_consensus.cc:729] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858abe7d735d4151b42343a729fc3f9e, State: Initialized, Role: FOLLOWER
I20260812 00:37:39.537343 6496 consensus_queue.cc:260] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:39.539361 6496 ts_tablet_manager.cc:1428] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
I20260812 00:37:39.539713 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:39.539170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:34367)
0812 00:37:39.539379 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:39.539513 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 00:37:39.540851 6713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:39.540423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:60537)
0812 00:37:39.540614 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:39.540709 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 00:37:39.883878 6715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:39.884364 6715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:39.884531 6715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:39.892318 6715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:39.892480 6715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:39.892644 6715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:37:39.903390 6715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:36563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=45335
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:39.904865 6715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:39.906473 6715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:39.920749 6725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:39.920820 6726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:39.920949 6728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:39.921563 6715 server_base.cc:1034] running on GCE node
I20260812 00:37:39.922305 6715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:39.923384 6715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:39.924631 6715 hybrid_clock.cc:648] HybridClock initialized: now 1786495059924563 us; error 62 us; skew 500 ppm
I20260812 00:37:39.924998 6715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:39 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495059, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:39.928498 6715 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:37:39.930179 6715 webserver.cc:460] Webserver started at http://127.7.244.66:45335/ using document root <none> and password file <none>
I20260812 00:37:39.930974 6715 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:39.931113 6715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:39.937402 6715 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:37:39.940585 6745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:39.942245 6715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:37:39.942456 6715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "482432aabf084ff5828eaa5f4915240a"
format_stamp: "Formatted at 2026-08-12 00:37:33 on dist-test-slave-gjw7"
server_key: "1677ef2d9f1b24495347bbd920bddf2b"
server_key_iv: "c0966c940467c4cf0adfff2a297bb747"
server_key_version: "encryptionkey@0"
I20260812 00:37:39.943564 6715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:39.956041 6715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:39.961184 6715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:39.961514 6715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:39.962617 6715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:39.964177 6715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:39.964294 6715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:39.964416 6715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:39.964489 6715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:39.996863 6715 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:36563
I20260812 00:37:39.996927 6980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:36563 every 8 connection(s)
I20260812 00:37:39.998579 6715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:37:39.999993 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 6715
I20260812 00:37:40.000732 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:36345
--local_ip_for_outbound_sockets=127.7.244.67
--tserver_master_addrs=127.7.244.126:39087
--webserver_port=33919
--webserver_interface=127.7.244.67
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:37:40.005271 6985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:40.002408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:57001)
0812 00:37:40.003440 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:40.003831 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":403,"threads_started":1}
W20260812 00:37:40.006176 6981 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:39087 (0 consecutive failures): Network error: Failed to ping master at 127.7.244.126:39087: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
I20260812 00:37:40.007421 6985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:40.006929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:54417)
0812 00:37:40.007140 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:40.007255 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 00:37:40.008553 6985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:40.008110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:52859)
0812 00:37:40.008279 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:40.008389 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 00:37:40.008885 6981 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:37:40.360776 6988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:40.361115 6988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:40.361214 6988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:40.369903 6988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:40.370041 6988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:40.370167 6988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:37:40.380466 6988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:36345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=33919
--tserver_master_addrs=127.7.244.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:40.382051 6988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:40.383466 6988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:40.398010 7006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:40.398007 7002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:40.398205 7008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:40.398921 6988 server_base.cc:1034] running on GCE node
I20260812 00:37:40.399659 6988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:40.400749 6988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:40.401957 6988 hybrid_clock.cc:648] HybridClock initialized: now 1786495060401891 us; error 65 us; skew 500 ppm
I20260812 00:37:40.402295 6988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:40 dist-test-slave-gjw7 krb5kdc[4176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495060, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:40.409258 6988 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:37:40.411098 6988 webserver.cc:460] Webserver started at http://127.7.244.67:33919/ using document root <none> and password file <none>
I20260812 00:37:40.411922 6988 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:40.412068 6988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:40.418563 6988 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:37:40.421804 7022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:40.423300 6988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:37:40.423538 6988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "07194dc481514a97835962d7f46b7d92"
format_stamp: "Formatted at 2026-08-12 00:37:34 on dist-test-slave-gjw7"
server_key: "0600622515e311130d87b31976d809a1"
server_key_iv: "4a9415f965500195c465aaab0c24bb27"
server_key_version: "encryptionkey@0"
I20260812 00:37:40.424604 6988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:40.445031 6988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:40.450134 6988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:40.450464 6988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:40.451633 6988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:40.453456 6988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:40.453585 6988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:40.453733 6988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:40.453817 6988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:40.488359 6988 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:36345
I20260812 00:37:40.488433 7246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:36345 every 8 connection(s)
I20260812 00:37:40.489965 6988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:37:40.492398 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 6988
I20260812 00:37:40.495484 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:40.493377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:41793)
0812 00:37:40.494245 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:40.494537 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":356,"threads_started":1}
W20260812 00:37:40.496205 7248 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:39087 (0 consecutive failures): Network error: Failed to ping master at 127.7.244.126:39087: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
I20260812 00:37:40.497152 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:40.496793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:53355)
0812 00:37:40.496951 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:40.497030 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 00:37:40.498087 7257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:40.497763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:33713)
0812 00:37:40.497888 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:40.497968 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 00:37:40.498346 7248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 00:37:40.542692 7260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:40.541833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:37315)
0812 00:37:40.542322 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:40.542455 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":229,"threads_started":1}
I20260812 00:37:41.010403 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:41.009532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:47407)
0812 00:37:41.010055 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:41.010146 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":245,"threads_started":1}
I20260812 00:37:41.297224 7265 raft_consensus.cc:482] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:37:41.297569 7265 raft_consensus.cc:504] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:41.299268 7265 leader_election.cc:288] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563), 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345)
Aug 12 00:37:41 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495059, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
Aug 12 00:37:41 dist-test-slave-gjw7 krb5kdc[4176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495059, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:37:41.341831 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:41.299634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:36563 (local address 127.7.244.65:53103)
0812 00:37:41.313958 (+ 14324us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:41.313980 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:41.314118 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:41.317997 (+ 3879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:41.318001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:41.319089 (+ 1088us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:41.320183 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:41.320218 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:41.322772 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:41.322779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:41.325321 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:41.325353 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:41.326938 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:41.326946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:41.327326 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:41.328262 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:41.328303 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:41.331472 (+ 3169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:41.336316 (+ 4844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:41.336329 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:41.336350 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:41.337518 (+ 1168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:41.338142 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:41.338147 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:41.338151 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:41.338537 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:41.339627 (+ 1090us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:41.339636 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:41.340220 (+ 584us) client_negotiation.cc:769] Sending connection context
0812 00:37:41.340606 (+ 386us) client_negotiation.cc:240] Negotiation successful
0812 00:37:41.341376 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14201,"mutex_wait_us":343,"spinlock_wait_cycles":94336,"thread_start_us":13205,"threads_started":1}
I20260812 00:37:41.342180 7272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:41.299908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:39299 (local address 127.7.244.67:36345)
0812 00:37:41.313658 (+ 13750us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:41.313675 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:41.313726 (+ 51us) server_negotiation.cc:407] Connection header received
0812 00:37:41.316535 (+ 2809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:41.316552 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:41.317164 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:41.317651 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:41.320487 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:41.322480 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:41.325537 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:41.326638 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:41.326877 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:41.332350 (+ 5473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:41.332362 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:41.332380 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:41.332431 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:41.336063 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:41.337727 (+ 1664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:41.337738 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:41.337747 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:41.338001 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:41.338668 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:41.338673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:41.338677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:41.339460 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:41.339656 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:41.341327 (+ 1671us) server_negotiation.cc:299] Negotiation successful
0812 00:37:41.341767 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13536,"thread_start_us":13404,"threads_started":1}
I20260812 00:37:41.342310 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:41.299956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:53103 (local address 127.7.244.66:36563)
0812 00:37:41.312338 (+ 12382us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:41.312361 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:41.314163 (+ 1802us) server_negotiation.cc:407] Connection header received
0812 00:37:41.316596 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:41.316607 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:41.317164 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:41.318067 (+ 903us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:41.320487 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:41.322480 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:41.325537 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:41.326638 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:41.326870 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:41.331885 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:41.331897 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:41.331915 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:41.331971 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:41.336063 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:41.337731 (+ 1668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:41.337740 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:41.337747 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:41.338001 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:41.338668 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:41.338673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:41.338677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:41.339460 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:41.339656 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:41.340987 (+ 1331us) server_negotiation.cc:299] Negotiation successful
0812 00:37:41.341602 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12093,"thread_start_us":12463,"threads_started":1}
I20260812 00:37:41.342630 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:41.299570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:36345 (local address 127.7.244.65:39299)
0812 00:37:41.313166 (+ 13596us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:41.313227 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:41.314018 (+ 791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:41.317939 (+ 3921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:41.317951 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:41.319233 (+ 1282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:41.320183 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:41.320217 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:41.322773 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:41.322779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:41.325321 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:41.325347 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:41.326905 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:41.326917 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:41.327324 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:41.328262 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:41.328303 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:41.331975 (+ 3672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:41.336316 (+ 4341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:41.336329 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:41.336350 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:41.337518 (+ 1168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:41.338141 (+ 623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:41.338147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:41.338151 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:41.338537 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:41.339627 (+ 1090us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:41.339636 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:41.340220 (+ 584us) client_negotiation.cc:769] Sending connection context
0812 00:37:41.340606 (+ 386us) client_negotiation.cc:240] Negotiation successful
0812 00:37:41.342274 (+ 1668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13438,"mutex_wait_us":1466,"spinlock_wait_cycles":35072,"thread_start_us":14083,"threads_started":1}
W20260812 00:37:41.346463 6486 leader_election.cc:341] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563): Not found: Tablet not found: 408ba4ca6a6549b19896b0ac57bd9e4d
W20260812 00:37:41.346861 6493 leader_election.cc:341] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345): Not found: Tablet not found: 408ba4ca6a6549b19896b0ac57bd9e4d
I20260812 00:37:41.347007 6493 leader_election.cc:302] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 858abe7d735d4151b42343a729fc3f9e; no voters: 07194dc481514a97835962d7f46b7d92, 482432aabf084ff5828eaa5f4915240a
I20260812 00:37:41.347460 7265 raft_consensus.cc:2726] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:37:41.499971 7274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:41.498988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:36567)
0812 00:37:41.499550 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:41.499710 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":280,"threads_started":1}
I20260812 00:37:41.544477 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:41.543896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:57699)
0812 00:37:41.544150 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:41.544279 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 00:37:42.012328 7276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:42.011536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:50387)
0812 00:37:42.012049 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:42.012146 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":273,"threads_started":1}
I20260812 00:37:42.501837 7279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:42.501004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:50359)
0812 00:37:42.501555 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:42.501649 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":331,"threads_started":1}
I20260812 00:37:42.546697 7280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:42.545698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:49289)
0812 00:37:42.546328 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:42.546444 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":297,"threads_started":1}
I20260812 00:37:43.014254 7289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:43.013410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:55587)
0812 00:37:43.013921 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:43.014034 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":271,"threads_started":1}
I20260812 00:37:43.265858 7293 raft_consensus.cc:482] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:37:43.266083 7293 raft_consensus.cc:504] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:43.266798 7293 leader_election.cc:288] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563), 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345)
W20260812 00:37:43.267850 6486 leader_election.cc:341] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563): Not found: Tablet not found: 408ba4ca6a6549b19896b0ac57bd9e4d
W20260812 00:37:43.268033 6493 leader_election.cc:341] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345): Not found: Tablet not found: 408ba4ca6a6549b19896b0ac57bd9e4d
I20260812 00:37:43.268190 6486 leader_election.cc:302] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 858abe7d735d4151b42343a729fc3f9e; no voters: 07194dc481514a97835962d7f46b7d92, 482432aabf084ff5828eaa5f4915240a
I20260812 00:37:43.268499 7293 raft_consensus.cc:2726] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:37:43.503799 7296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:43.502884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:50557)
0812 00:37:43.503530 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:43.503620 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":378,"threads_started":1}
I20260812 00:37:43.548964 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:43.547952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:34153)
0812 00:37:43.548540 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:43.548711 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":302,"threads_started":1}
I20260812 00:37:44.016153 7299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:44.015345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:59091)
0812 00:37:44.015831 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:44.015953 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":218,"threads_started":1}
I20260812 00:37:44.505825 7301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:44.504889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:59337)
0812 00:37:44.505463 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:44.505580 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":248,"threads_started":1}
I20260812 00:37:44.551062 7302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:44.550270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:52607)
0812 00:37:44.550760 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:44.550856 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":243,"threads_started":1}
I20260812 00:37:45.018098 7303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:45.017253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:36595)
0812 00:37:45.017821 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:45.017918 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":298,"threads_started":1}
I20260812 00:37:45.507815 7305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:45.507012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:36475)
0812 00:37:45.507528 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:45.507636 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":267,"threads_started":1}
I20260812 00:37:45.552928 7306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:45.552107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:35811)
0812 00:37:45.552622 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:45.552739 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":275,"threads_started":1}
I20260812 00:37:45.840884 7308 raft_consensus.cc:482] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:37:45.841099 7308 raft_consensus.cc:504] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:45.841909 7308 leader_election.cc:288] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563), 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345)
W20260812 00:37:45.842948 6486 leader_election.cc:341] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563): Not found: Tablet not found: 408ba4ca6a6549b19896b0ac57bd9e4d
W20260812 00:37:45.843132 6493 leader_election.cc:341] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345): Not found: Tablet not found: 408ba4ca6a6549b19896b0ac57bd9e4d
I20260812 00:37:45.843248 6493 leader_election.cc:302] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 858abe7d735d4151b42343a729fc3f9e; no voters: 07194dc481514a97835962d7f46b7d92, 482432aabf084ff5828eaa5f4915240a
I20260812 00:37:45.843516 7308 raft_consensus.cc:2726] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:37:46.019793 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:46.018977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:40465)
0812 00:37:46.019501 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:46.019599 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":270,"threads_started":1}
I20260812 00:37:46.509617 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:46.508782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:54927)
0812 00:37:46.509328 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:46.509433 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":261,"threads_started":1}
I20260812 00:37:46.554637 7313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:46.553868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:33541)
0812 00:37:46.554345 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:46.554433 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":249,"threads_started":1}
I20260812 00:37:47.021996 7318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:47.020899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:47535)
0812 00:37:47.021600 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:47.021734 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":323,"threads_started":1}
I20260812 00:37:47.511584 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:47.510667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:37649)
0812 00:37:47.511276 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:47.511385 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":276,"threads_started":1}
I20260812 00:37:47.556577 7321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:47.555740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:49713)
0812 00:37:47.556300 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:47.556400 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":289,"threads_started":1}
I20260812 00:37:48.023948 7323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:48.023098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:59319)
0812 00:37:48.023622 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:48.023729 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":256,"threads_started":1}
I20260812 00:37:48.513615 7327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:48.512679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:55413)
0812 00:37:48.513220 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:48.513380 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":268,"threads_started":1}
I20260812 00:37:48.558611 7330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:48.557778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:48325)
0812 00:37:48.558287 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:48.558422 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":244,"threads_started":1}
I20260812 00:37:48.597218 7331 raft_consensus.cc:482] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:37:48.597533 7331 raft_consensus.cc:504] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482432aabf084ff5828eaa5f4915240a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 36563 } } peers { permanent_uuid: "07194dc481514a97835962d7f46b7d92" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 36345 } } peers { permanent_uuid: "858abe7d735d4151b42343a729fc3f9e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38521 } }
I20260812 00:37:48.598279 7331 leader_election.cc:288] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563), 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345)
W20260812 00:37:48.599191 6486 leader_election.cc:341] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 482432aabf084ff5828eaa5f4915240a (127.7.244.66:36563): Not found: Tablet not found: 408ba4ca6a6549b19896b0ac57bd9e4d
W20260812 00:37:48.599354 6493 leader_election.cc:341] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 07194dc481514a97835962d7f46b7d92 (127.7.244.67:36345): Not found: Tablet not found: 408ba4ca6a6549b19896b0ac57bd9e4d
I20260812 00:37:48.599496 6493 leader_election.cc:302] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 858abe7d735d4151b42343a729fc3f9e; no voters: 07194dc481514a97835962d7f46b7d92, 482432aabf084ff5828eaa5f4915240a
I20260812 00:37:48.599752 7331 raft_consensus.cc:2726] T 408ba4ca6a6549b19896b0ac57bd9e4d P 858abe7d735d4151b42343a729fc3f9e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:37:49.025967 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:49.025056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:56667)
0812 00:37:49.025681 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:49.025786 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":336,"threads_started":1}
I20260812 00:37:49.515432 7334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:49.514645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:40171)
0812 00:37:49.515132 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:49.515260 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":225,"threads_started":1}
I20260812 00:37:49.560729 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:49.559833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:45711)
0812 00:37:49.560428 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:49.560540 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":285,"threads_started":1}
I20260812 00:37:50.028211 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:50.027350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:36865)
0812 00:37:50.027886 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:50.028013 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":286,"threads_started":1}
I20260812 00:37:50.517198 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:50.516413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:44841)
0812 00:37:50.516903 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:50.517001 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":253,"threads_started":1}
I20260812 00:37:50.562753 7339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:50.561921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:36591)
0812 00:37:50.562446 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:50.562553 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":277,"threads_started":1}
I20260812 00:37:51.030210 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:51.029430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:42903)
0812 00:37:51.029942 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:51.030035 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":232,"threads_started":1}
I20260812 00:37:51.518973 7344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:51.518153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:49691)
0812 00:37:51.518684 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:51.518780 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":281,"threads_started":1}
I20260812 00:37:51.564587 7346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:51.563799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:49345)
0812 00:37:51.564282 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:51.564381 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":251,"threads_started":1}
I20260812 00:37:52.032008 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:52.031190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:47887)
0812 00:37:52.031728 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:52.031829 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":303,"threads_started":1}
I20260812 00:37:52.521050 7350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:52.520173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:36115)
0812 00:37:52.520760 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:52.520856 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":337,"threads_started":1}
I20260812 00:37:52.566627 7352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:52.565687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:59761)
0812 00:37:52.566324 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:52.566441 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":386,"threads_started":1}
I20260812 00:37:53.033830 7353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:53.033003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:51547)
0812 00:37:53.033559 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:53.033663 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":316,"threads_started":1}
I20260812 00:37:53.523094 7359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:53.522228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:38655)
0812 00:37:53.522747 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:53.522865 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":247,"threads_started":1}
I20260812 00:37:53.568656 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:53.567757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:35101)
0812 00:37:53.568287 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:53.568414 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":250,"threads_started":1}
I20260812 00:37:54.035796 7361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:54.034911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:56051)
0812 00:37:54.035492 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:54.035595 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":286,"threads_started":1}
I20260812 00:37:54.525162 7362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:54.524235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:43379)
0812 00:37:54.524794 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:54.524934 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":279,"threads_started":1}
I20260812 00:37:54.570904 7363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:54.570106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.65:49527)
0812 00:37:54.570610 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:54.570714 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":249,"threads_started":1}
I20260812 00:37:55.038168 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:55.037284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.66:40961)
0812 00:37:55.037875 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:55.037989 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":261,"threads_started":1}
I20260812 00:37:55.495649 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 6400
I20260812 00:37:55.518633 7367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:55.496974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.7.244.65:38521)
0812 00:37:55.497786 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:55.497796 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:55.497828 (+ 32us) server_negotiation.cc:407] Connection header received
0812 00:37:55.497904 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:55.497913 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:55.498129 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:55.498280 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:55.500453 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:55.501557 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:55.502999 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:55.503651 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:55.503771 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:55.511830 (+ 8059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:55.511850 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:55.511869 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:55.511913 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:55.515437 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:55.516306 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:55.516315 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:55.516318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:55.516475 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:55.516895 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:55.516900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:55.516902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:55.517345 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:55.517675 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:55.517990 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 00:37:55.518281 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":511,"threads_started":1}
I20260812 00:37:55.520028 6698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:55.527180 7379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:55.526372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:39087 (local address 127.7.244.67:48171)
0812 00:37:55.526893 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:55.527011 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":253,"threads_started":1}
I20260812 00:37:55.611543 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 6400
I20260812 00:37:55.640977 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 6715
I20260812 00:37:55.655754 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:55.641856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.7.244.66:36563)
0812 00:37:55.642366 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:55.642370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:55.642388 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:37:55.642453 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:55.642457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:55.642520 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:55.642614 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:55.644044 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:55.644891 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:55.646342 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:55.646802 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:55.646936 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:55.649898 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:55.649919 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:55.649922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:55.649965 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:55.652742 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:55.653757 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:55.653761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:55.653764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:55.653940 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:55.654357 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:55.654360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:55.654362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:55.654665 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:55.654879 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:55.655197 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 00:37:55.655397 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":292,"threads_started":1}
I20260812 00:37:55.656637 6975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:55.740767 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 6715
I20260812 00:37:55.772725 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 6988
I20260812 00:37:55.787842 7385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:55.773594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.7.244.67:36345)
0812 00:37:55.774141 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:55.774146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:55.774166 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:37:55.774228 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:55.774232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:55.774304 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:55.774414 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:55.775888 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:55.776732 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:55.778252 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:55.778724 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:55.778832 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:55.781720 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:55.781725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:55.781729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:55.781772 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:55.784783 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:55.785828 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:55.785833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:55.785836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:55.786030 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:55.786390 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:55.786393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:55.786395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:55.786687 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:55.786928 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:55.787256 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 00:37:55.787507 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":265,"threads_started":1}
I20260812 00:37:55.788605 7240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:37:55.872092 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 6988
2026-08-12T00:37:55Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25584 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:37:55 dist-test-slave-gjw7 krb5kdc[7390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:37:55 dist-test-slave-gjw7 krb5kdc[7390](info): set up 2 sockets
Aug 12 00:37:55 dist-test-slave-gjw7 krb5kdc[7390](info): commencing operation
krb5kdc: starting...
W20260812 00:37:57.221813 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:37:57 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:37:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:37:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:37:57.526947 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42765
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:57.885740 7411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:57.886071 7411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:57.886163 7411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:57.893748 7411 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 00:37:57.893873 7411 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 00:37:57.893913 7411 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 00:37:57.893977 7411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:37:57.894032 7411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:57.894100 7411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:57.894143 7411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:37:57.894196 7411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:37:57.904748 7411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:57.906250 7411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:57.907696 7411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:57.920933 7430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:57.920948 7427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:57.921314 7411 server_base.cc:1034] running on GCE node
W20260812 00:37:57.921814 7424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:57.922775 7411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:57.924319 7411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:57.925559 7411 hybrid_clock.cc:648] HybridClock initialized: now 1786495077925481 us; error 48 us; skew 500 ppm
I20260812 00:37:57.925957 7411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:57 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495077, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:57.929356 7411 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:37:57.931051 7411 webserver.cc:460] Webserver started at http://127.7.244.126:39659/ using document root <none> and password file <none>
I20260812 00:37:57.931833 7411 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:57.931946 7411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:57.932329 7411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:57.935842 7411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "f83e42a0f656413aa42c9fd1921afef5"
format_stamp: "Formatted at 2026-08-12 00:37:57 on dist-test-slave-gjw7"
server_key: "d36733e8f8f3d1231eef6301e7c586c2"
server_key_iv: "7168239da53afccdc522311edecbc222"
server_key_version: "encryptionkey@0"
I20260812 00:37:57.936892 7411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "f83e42a0f656413aa42c9fd1921afef5"
format_stamp: "Formatted at 2026-08-12 00:37:57 on dist-test-slave-gjw7"
server_key: "d36733e8f8f3d1231eef6301e7c586c2"
server_key_iv: "7168239da53afccdc522311edecbc222"
server_key_version: "encryptionkey@0"
I20260812 00:37:57.943346 7411 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 00:37:57.947379 7444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:57.948911 7411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:37:57.949118 7411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "f83e42a0f656413aa42c9fd1921afef5"
format_stamp: "Formatted at 2026-08-12 00:37:57 on dist-test-slave-gjw7"
server_key: "d36733e8f8f3d1231eef6301e7c586c2"
server_key_iv: "7168239da53afccdc522311edecbc222"
server_key_version: "encryptionkey@0"
I20260812 00:37:57.949405 7411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:57.974097 7411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:57.978545 7411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:57.978833 7411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:57.997905 7411 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:42765
I20260812 00:37:57.997910 7558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:42765 every 8 connection(s)
I20260812 00:37:57.999452 7411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:37:58.003746 7559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:58.007776 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 7411
I20260812 00:37:58.008008 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:37:58.008661 8145 external_mini_cluster.cc:1428] Setting key f94d19c2d2d9fb0934c5492bcdeface8
I20260812 00:37:58.018443 7559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Bootstrap starting.
I20260812 00:37:58.022542 7559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:58.023679 7559 log.cc:824] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:37:58 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:58.027215 7559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: No bootstrap required, opened a new log
I20260812 00:37:58.035961 7563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:58.011011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.7.244.126:42765)
0812 00:37:58.011908 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:58.011927 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:58.011985 (+ 58us) server_negotiation.cc:407] Connection header received
0812 00:37:58.013822 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:58.013862 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:58.014496 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:58.015035 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:58.016543 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:58.018621 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:58.020109 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:58.021266 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:58.021530 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:58.025933 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:58.025946 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:58.025967 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:58.026016 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:58.030448 (+ 4432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:58.031404 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:58.031416 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:58.031427 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:58.031676 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:58.032080 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:58.032086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:58.032089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:58.032965 (+ 876us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:58.033347 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:58.033825 (+ 478us) server_negotiation.cc:299] Negotiation successful
0812 00:37:58.034382 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":383,"threads_started":1}
I20260812 00:37:58.037909 7559 raft_consensus.cc:348] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:37:58.038290 7559 raft_consensus.cc:374] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:58.038408 7559 raft_consensus.cc:729] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f83e42a0f656413aa42c9fd1921afef5, State: Initialized, Role: FOLLOWER
I20260812 00:37:58.039029 7559 consensus_queue.cc:260] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:37:58.039351 7559 raft_consensus.cc:388] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:37:58.039517 7559 raft_consensus.cc:482] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:37:58.039690 7559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:37:58.042394 7559 raft_consensus.cc:504] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:37:58.042999 7559 leader_election.cc:302] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f83e42a0f656413aa42c9fd1921afef5; no voters:
I20260812 00:37:58.043654 7559 leader_election.cc:288] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:37:58.043859 7576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:37:58.045342 7576 raft_consensus.cc:686] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 1 LEADER]: Becoming Leader. State: Replica: f83e42a0f656413aa42c9fd1921afef5, State: Running, Role: LEADER
I20260812 00:37:58.046002 7576 consensus_queue.cc:237] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:37:58.046298 7559 sys_catalog.cc:564] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:37:58.049747 7578 sys_catalog.cc:455] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f83e42a0f656413aa42c9fd1921afef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } } }
I20260812 00:37:58.050240 7578 sys_catalog.cc:458] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:58.050796 7581 sys_catalog.cc:455] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f83e42a0f656413aa42c9fd1921afef5. Latest consensus state: current_term: 1 leader_uuid: "f83e42a0f656413aa42c9fd1921afef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } } }
I20260812 00:37:58.051167 7581 sys_catalog.cc:458] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:37:58.051287 7590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:37:58.057952 7590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:37:58.068032 7590 catalog_manager.cc:1347] Generated new cluster ID: 316f0d0dac02472f9be83f8abdfc61cd
I20260812 00:37:58.068251 7590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:37:58.094079 7590 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:37:58.095474 7590 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:37:58.104257 7590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Generated new TSK 0
I20260812 00:37:58.105654 7590 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:37:58.189603 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42765
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:58.551842 7631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:58.552253 7631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:58.552374 7631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:58.560151 7631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:58.560278 7631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:58.560402 7631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:37:58.570593 7631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:58.571944 7631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:58.573383 7631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:58.587286 7641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:58.587251 7645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:58.587316 7642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:58.588687 7631 server_base.cc:1034] running on GCE node
I20260812 00:37:58.589425 7631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:58.590457 7631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:58.591667 7631 hybrid_clock.cc:648] HybridClock initialized: now 1786495078591614 us; error 54 us; skew 500 ppm
I20260812 00:37:58.591990 7631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:58 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495078, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:58.595265 7631 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:37:58.596800 7631 webserver.cc:460] Webserver started at http://127.7.244.65:34173/ using document root <none> and password file <none>
I20260812 00:37:58.597584 7631 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:58.597694 7631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:58.597981 7631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:58.601068 7631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "da292be6ba9445d180916e1368fe48bf"
format_stamp: "Formatted at 2026-08-12 00:37:58 on dist-test-slave-gjw7"
server_key: "254ce4dea87055d656ea9752b60f2e20"
server_key_iv: "18c657a391d6f3780dab7650ec63e32c"
server_key_version: "encryptionkey@0"
I20260812 00:37:58.601943 7631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "da292be6ba9445d180916e1368fe48bf"
format_stamp: "Formatted at 2026-08-12 00:37:58 on dist-test-slave-gjw7"
server_key: "254ce4dea87055d656ea9752b60f2e20"
server_key_iv: "18c657a391d6f3780dab7650ec63e32c"
server_key_version: "encryptionkey@0"
I20260812 00:37:58.607694 7631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 00:37:58.611768 7657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:58.613353 7631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:37:58.613523 7631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "da292be6ba9445d180916e1368fe48bf"
format_stamp: "Formatted at 2026-08-12 00:37:58 on dist-test-slave-gjw7"
server_key: "254ce4dea87055d656ea9752b60f2e20"
server_key_iv: "18c657a391d6f3780dab7650ec63e32c"
server_key_version: "encryptionkey@0"
I20260812 00:37:58.613714 7631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:58.632210 7631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:58.636793 7631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:58.637101 7631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:58.638165 7631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:58.639634 7631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:58.639711 7631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:58.639837 7631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:58.639909 7631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:58.671530 7631 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:33587
I20260812 00:37:58.671582 7913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:33587 every 8 connection(s)
I20260812 00:37:58.672902 7631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:37:58.682338 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 7631
I20260812 00:37:58.682562 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:37:58.683270 8145 external_mini_cluster.cc:1428] Setting key 0f66cef4825a7ffc7cc0bd789c25040a
Aug 12 00:37:58 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495078, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:58.693500 7919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:58.675623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:36289 (local address 127.7.244.126:42765)
0812 00:37:58.676066 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:58.676070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:58.677218 (+ 1148us) server_negotiation.cc:407] Connection header received
0812 00:37:58.679184 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:58.679190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:58.679288 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:58.679389 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:58.681059 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:58.681858 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:58.683573 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:58.683999 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:58.684115 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:58.687786 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:58.687792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:58.687795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:58.687830 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:58.690309 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:58.691092 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:58.691097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:58.691099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:58.691228 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:58.691659 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:58.691663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:58.691665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:58.691958 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:58.692095 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:58.692870 (+ 775us) server_negotiation.cc:299] Negotiation successful
0812 00:37:58.693108 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":207,"threads_started":1}
I20260812 00:37:58.694533 7922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:58.676245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:36289)
0812 00:37:58.677017 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:58.677069 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:58.678878 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:58.679667 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:58.679676 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:58.680249 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:58.680868 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:58.680886 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:58.682033 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:58.682038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:58.683420 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:58.683436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:58.684244 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:58.684253 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:58.684494 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:58.685230 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:58.685333 (+ 103us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:58.687535 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:58.690470 (+ 2935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:58.690480 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:58.690499 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:58.690964 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:58.691340 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:58.691345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:58.691349 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:58.691549 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:58.692087 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:58.692096 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:58.692484 (+ 388us) client_negotiation.cc:769] Sending connection context
0812 00:37:58.692745 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 00:37:58.693343 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":293,"threads_started":1}
I20260812 00:37:58.696187 7916 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42765
I20260812 00:37:58.696506 7916 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:58.697485 7916 heartbeater.cc:511] Master 127.7.244.126:42765 requested a full tablet report, sending...
I20260812 00:37:58.700229 7478 ts_manager.cc:194] Registered new tserver with Master: da292be6ba9445d180916e1368fe48bf (127.7.244.65:33587)
I20260812 00:37:58.702323 7478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:36289
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:37:58.752746 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42765
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:59.108821 7925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:59.109162 7925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:59.109338 7925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:59.116875 7925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:59.117070 7925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:59.117278 7925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:37:59.127779 7925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:59.129153 7925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:59.130599 7925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:59.145674 7936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:59.145625 7932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:59.145702 7943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:59.146641 7925 server_base.cc:1034] running on GCE node
I20260812 00:37:59.147370 7925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:59.148485 7925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:59.149727 7925 hybrid_clock.cc:648] HybridClock initialized: now 1786495079149666 us; error 55 us; skew 500 ppm
I20260812 00:37:59.150079 7925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:59 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495079, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:59.153649 7925 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:37:59.155431 7925 webserver.cc:460] Webserver started at http://127.7.244.66:33873/ using document root <none> and password file <none>
I20260812 00:37:59.156224 7925 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:59.156369 7925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:59.156708 7925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:59.160015 7925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "39d5c5787a464663aab8ee580d96e0f2"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "21b08c193b95a4b2bd0269714d558298"
server_key_iv: "1a86e1c86d70f62b7d56cf297b069762"
server_key_version: "encryptionkey@0"
I20260812 00:37:59.160889 7925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "39d5c5787a464663aab8ee580d96e0f2"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "21b08c193b95a4b2bd0269714d558298"
server_key_iv: "1a86e1c86d70f62b7d56cf297b069762"
server_key_version: "encryptionkey@0"
I20260812 00:37:59.167068 7925 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260812 00:37:59.171228 7957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:59.172808 7925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 00:37:59.172997 7925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "39d5c5787a464663aab8ee580d96e0f2"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "21b08c193b95a4b2bd0269714d558298"
server_key_iv: "1a86e1c86d70f62b7d56cf297b069762"
server_key_version: "encryptionkey@0"
I20260812 00:37:59.173213 7925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:59.189924 7925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:59.194489 7925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:59.194778 7925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:59.195852 7925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:59.197329 7925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:59.197413 7925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:59.197531 7925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:59.197587 7925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:59.230465 7925 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:37911
I20260812 00:37:59.230489 8199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:37911 every 8 connection(s)
I20260812 00:37:59.231880 7925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:37:59.232775 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 7925
I20260812 00:37:59.233008 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:37:59.233705 8145 external_mini_cluster.cc:1428] Setting key 0b9aa63311bf8e989728435b677fa8b2
Aug 12 00:37:59 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495079, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:59.256402 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.235002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:53983 (local address 127.7.244.126:42765)
0812 00:37:59.235523 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:59.235530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:59.236525 (+ 995us) server_negotiation.cc:407] Connection header received
0812 00:37:59.238585 (+ 2060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:59.238591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:59.238698 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:59.238830 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:59.240519 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.241593 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.243492 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.244163 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.244332 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.248788 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:59.248795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:59.248799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:59.248842 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:59.251730 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:59.253080 (+ 1350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:59.253086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:59.253088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:59.253270 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:59.254177 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:59.254183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:59.254185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:59.254629 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:59.254759 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:59.255823 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0812 00:37:59.256059 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":253,"threads_started":1}
I20260812 00:37:59.257937 8203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.235417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:53983)
0812 00:37:59.236331 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:59.236389 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:59.238238 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:59.239068 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:59.239077 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:59.239647 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:59.240289 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.240310 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.241906 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.241911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.243277 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.243295 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.244458 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.244464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.244708 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.245549 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:59.245575 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:59.248394 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:59.252038 (+ 3644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:59.252050 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:59.252072 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:59.252886 (+ 814us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:59.253507 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:59.253513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:59.253517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:59.253980 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:59.254788 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:59.254800 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:59.255325 (+ 525us) client_negotiation.cc:769] Sending connection context
0812 00:37:59.255703 (+ 378us) client_negotiation.cc:240] Negotiation successful
0812 00:37:59.256345 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":374,"threads_started":1}
I20260812 00:37:59.259673 8200 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42765
I20260812 00:37:59.260035 8200 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:59.260905 8200 heartbeater.cc:511] Master 127.7.244.126:42765 requested a full tablet report, sending...
I20260812 00:37:59.262651 7478 ts_manager.cc:194] Registered new tserver with Master: 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911)
I20260812 00:37:59.263541 7478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:53983
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:37:59.301807 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42765
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:37:59.652863 8209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:37:59.653213 8209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:37:59.653386 8209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:37:59.660923 8209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:37:59.661055 8209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:37:59.661173 8209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:37:59.671263 8209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:37:59.672622 8209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:37:59.674010 8209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:37:59.687815 8226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:59.687888 8231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:37:59.687888 8221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:37:59.688570 8209 server_base.cc:1034] running on GCE node
I20260812 00:37:59.689335 8209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:37:59.690426 8209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:37:59.691634 8209 hybrid_clock.cc:648] HybridClock initialized: now 1786495079691579 us; error 53 us; skew 500 ppm
I20260812 00:37:59.691957 8209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:37:59 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495079, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:37:59.695259 8209 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:37:59.696853 8209 webserver.cc:460] Webserver started at http://127.7.244.67:42033/ using document root <none> and password file <none>
I20260812 00:37:59.697669 8209 fs_manager.cc:359] Metadata directory not provided
I20260812 00:37:59.697788 8209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:37:59.698081 8209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:37:59.701215 8209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "7bfba26eb500425a95e74f89672e3f33"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "e0f3bc8b28139f04bcb00d9956db082b"
server_key_iv: "06b7400af251f8d900e6cda513a53e01"
server_key_version: "encryptionkey@0"
I20260812 00:37:59.702133 8209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "7bfba26eb500425a95e74f89672e3f33"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "e0f3bc8b28139f04bcb00d9956db082b"
server_key_iv: "06b7400af251f8d900e6cda513a53e01"
server_key_version: "encryptionkey@0"
I20260812 00:37:59.706537 7916 heartbeater.cc:503] Master 127.7.244.126:42765 was elected leader, sending a full tablet report...
I20260812 00:37:59.708173 8209 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 00:37:59.712219 8243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:59.713920 8209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:37:59.714123 8209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "7bfba26eb500425a95e74f89672e3f33"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "e0f3bc8b28139f04bcb00d9956db082b"
server_key_iv: "06b7400af251f8d900e6cda513a53e01"
server_key_version: "encryptionkey@0"
I20260812 00:37:59.714344 8209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:37:59.736091 8209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:37:59.740711 8209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:37:59.741014 8209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:37:59.742100 8209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:37:59.743587 8209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:37:59.743671 8209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:59.743801 8209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:37:59.743856 8209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:37:59.776080 8209 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:34625
I20260812 00:37:59.776118 8455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:34625 every 8 connection(s)
I20260812 00:37:59.777575 8209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:37:59.781411 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 8209
I20260812 00:37:59.781612 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:37:59.782083 8145 external_mini_cluster.cc:1428] Setting key cad996a10239b52e969a27b37cf12201
Aug 12 00:37:59 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495079, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:37:59.800408 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.780387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:49467 (local address 127.7.244.126:42765)
0812 00:37:59.780826 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:59.780831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:59.782112 (+ 1281us) server_negotiation.cc:407] Connection header received
0812 00:37:59.784287 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:59.784293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:59.784402 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:59.784521 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:59.786637 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.787547 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.789162 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.789624 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.789724 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.793504 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:59.793510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:59.793514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:59.793554 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:59.796314 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:59.797464 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:59.797470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:59.797473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:59.797652 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:59.798211 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:59.798217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:59.798220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:59.798771 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:59.798957 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:59.799856 (+ 899us) server_negotiation.cc:299] Negotiation successful
0812 00:37:59.800106 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":211,"threads_started":1}
I20260812 00:37:59.801425 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.780970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:49467)
0812 00:37:59.781894 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:59.781944 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:59.783939 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:59.784862 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:59.784889 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:59.785757 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:37:59.786447 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.786471 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.787721 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.787726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.789031 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.789045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.789856 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.789861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.790099 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.790799 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:37:59.790830 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:37:59.793180 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:37:59.796542 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:59.796554 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:59.796574 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:59.797281 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:59.797774 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:37:59.797779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:37:59.797782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:37:59.798040 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:37:59.798941 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:37:59.798952 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:37:59.799443 (+ 491us) client_negotiation.cc:769] Sending connection context
0812 00:37:59.799702 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 00:37:59.800166 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":398,"threads_started":1}
I20260812 00:37:59.802999 8456 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42765
I20260812 00:37:59.803331 8456 heartbeater.cc:464] Registering TS with master...
I20260812 00:37:59.804140 8456 heartbeater.cc:511] Master 127.7.244.126:42765 requested a full tablet report, sending...
I20260812 00:37:59.805809 7478 ts_manager.cc:194] Registered new tserver with Master: 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625)
I20260812 00:37:59.806668 7478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:49467
I20260812 00:37:59.811872 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:37:59.829593 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.816460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.7.244.126:42765)
0812 00:37:59.816628 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:59.816634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:59.817087 (+ 453us) server_negotiation.cc:407] Connection header received
0812 00:37:59.817339 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:59.817345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:59.817447 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:59.817533 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:37:59.819268 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.820045 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.821536 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.821970 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.822067 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.824518 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:37:59.824523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:37:59.824527 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:37:59.824560 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:37:59.827124 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:59.827841 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:59.827845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:59.827847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:59.827984 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:37:59.828273 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:37:59.828276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:37:59.828278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:37:59.828629 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:37:59.828817 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:59.829096 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 00:37:59.829280 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 00:37:59.834064 7478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:37:59.841090 7478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:37:59.868016 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.855186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:36714)
0812 00:37:59.855994 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:59.856034 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:59.856217 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:59.858113 (+ 1896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:59.858122 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:59.858156 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:59.858613 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.858630 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.861193 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.861202 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.864043 (+ 2841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.864081 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.866682 (+ 2601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.866692 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.866996 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.867076 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 00:37:59.867233 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 00:37:59.867500 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":387,"threads_started":1}
I20260812 00:37:59.870179 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.855291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36714 (local address 127.7.244.65:33587)
0812 00:37:59.857453 (+ 2162us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:59.857463 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:59.857495 (+ 32us) server_negotiation.cc:407] Connection header received
0812 00:37:59.857603 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:59.857609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:59.857853 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:59.858866 (+ 1013us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:59.859175 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.860971 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.864429 (+ 3458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.866313 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.868772 (+ 2459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.869135 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:59.869237 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 00:37:59.869754 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1938,"thread_start_us":1853,"threads_started":1}
I20260812 00:37:59.875041 7757 tablet_service.cc:1467] Processing CreateTablet for tablet 44e85aba683548b488a3ee3f0dc99b12 (DEFAULT_TABLE table=test-table [id=1ababe758d5e4d07b2f292a899f6002e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:59.876782 7757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e85aba683548b488a3ee3f0dc99b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:59.878413 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.857390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:34625 (local address 127.0.0.1:60534)
0812 00:37:59.868411 (+ 11021us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:59.868435 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:59.868563 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:59.871840 (+ 3277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:59.871844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:59.871875 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:59.872097 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.872110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.873955 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.873961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.875380 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.875398 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.877461 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.877468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.877684 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.877723 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 00:37:59.877832 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 00:37:59.878030 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10853,"mutex_wait_us":50,"thread_start_us":11051,"threads_started":1}
I20260812 00:37:59.879467 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.856707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:37911 (local address 127.0.0.1:37622)
0812 00:37:59.857316 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 00:37:59.857337 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:37:59.857461 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:37:59.870239 (+ 12778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:37:59.870243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:37:59.870273 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:37:59.870581 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.870598 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.873318 (+ 2720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.873324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.875534 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:37:59.875553 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.878585 (+ 3032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.878592 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:37:59.878777 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.878812 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:37:59.878905 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 00:37:59.879057 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":5,"thread_start_us":296,"threads_started":1}
I20260812 00:37:59.879812 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.856961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37622 (local address 127.7.244.66:37911)
0812 00:37:59.869478 (+ 12517us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:59.869489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:59.869526 (+ 37us) server_negotiation.cc:407] Connection header received
0812 00:37:59.869661 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:59.869671 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:59.869982 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:59.870242 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:59.871501 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.873096 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.875734 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.878078 (+ 2344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.878222 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.878436 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:59.879035 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 00:37:59.879335 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12281,"thread_start_us":13600,"threads_started":1}
I20260812 00:37:59.880851 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:37:59.858038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.7.244.67:34625)
0812 00:37:59.871312 (+ 13274us) server_negotiation.cc:206] Beginning negotiation
0812 00:37:59.871319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:37:59.871349 (+ 30us) server_negotiation.cc:407] Connection header received
0812 00:37:59.871432 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:37:59.871438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:37:59.871626 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:37:59.871812 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:37:59.872290 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.873757 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.875575 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:37:59.877201 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:37:59.877408 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:37:59.877695 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0812 00:37:59.880297 (+ 2602us) server_negotiation.cc:299] Negotiation successful
0812 00:37:59.880519 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13110,"thread_start_us":12948,"threads_started":1}
I20260812 00:37:59.885245 8048 tablet_service.cc:1467] Processing CreateTablet for tablet 44e85aba683548b488a3ee3f0dc99b12 (DEFAULT_TABLE table=test-table [id=1ababe758d5e4d07b2f292a899f6002e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:59.885245 8331 tablet_service.cc:1467] Processing CreateTablet for tablet 44e85aba683548b488a3ee3f0dc99b12 (DEFAULT_TABLE table=test-table [id=1ababe758d5e4d07b2f292a899f6002e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:37:59.887063 8048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e85aba683548b488a3ee3f0dc99b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:59.887058 8331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e85aba683548b488a3ee3f0dc99b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:37:59.893328 8497 tablet_bootstrap.cc:492] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: Bootstrap starting.
I20260812 00:37:59.897162 8497 tablet_bootstrap.cc:654] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:59.898699 8497 log.cc:824] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:59.901703 8497 tablet_bootstrap.cc:492] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: No bootstrap required, opened a new log
I20260812 00:37:59.902005 8497 ts_tablet_manager.cc:1397] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260812 00:37:59.902952 8501 tablet_bootstrap.cc:492] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Bootstrap starting.
I20260812 00:37:59.904353 8503 tablet_bootstrap.cc:492] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Bootstrap starting.
I20260812 00:37:59.907898 8501 tablet_bootstrap.cc:654] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:59.908315 8503 tablet_bootstrap.cc:654] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Neither blocks nor log segments found. Creating new log.
I20260812 00:37:59.909082 8501 log.cc:824] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:59.909665 8503 log.cc:824] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Log is configured to *not* fsync() on all Append() calls
I20260812 00:37:59.912997 8503 tablet_bootstrap.cc:492] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: No bootstrap required, opened a new log
I20260812 00:37:59.913058 8501 tablet_bootstrap.cc:492] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: No bootstrap required, opened a new log
I20260812 00:37:59.913335 8503 ts_tablet_manager.cc:1397] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 00:37:59.913378 8501 ts_tablet_manager.cc:1397] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260812 00:37:59.913056 8497 raft_consensus.cc:348] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:37:59.913671 8497 raft_consensus.cc:374] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:59.913771 8497 raft_consensus.cc:729] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da292be6ba9445d180916e1368fe48bf, State: Initialized, Role: FOLLOWER
I20260812 00:37:59.914561 8497 consensus_queue.cc:260] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:37:59.922394 8503 raft_consensus.cc:348] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:37:59.922828 8503 raft_consensus.cc:374] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:59.922938 8503 raft_consensus.cc:729] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bfba26eb500425a95e74f89672e3f33, State: Initialized, Role: FOLLOWER
I20260812 00:37:59.923597 8503 consensus_queue.cc:260] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:37:59.925066 8501 raft_consensus.cc:348] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:37:59.925717 8501 raft_consensus.cc:374] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:37:59.925880 8501 raft_consensus.cc:729] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d5c5787a464663aab8ee580d96e0f2, State: Initialized, Role: FOLLOWER
I20260812 00:37:59.926744 8501 consensus_queue.cc:260] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:37:59.927027 8497 ts_tablet_manager.cc:1428] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: Time spent starting tablet: real 0.025s user 0.010s sys 0.013s
I20260812 00:37:59.928372 8456 heartbeater.cc:503] Master 127.7.244.126:42765 was elected leader, sending a full tablet report...
I20260812 00:37:59.928709 8200 heartbeater.cc:503] Master 127.7.244.126:42765 was elected leader, sending a full tablet report...
I20260812 00:37:59.928977 8503 ts_tablet_manager.cc:1428] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260812 00:37:59.930404 8501 ts_tablet_manager.cc:1428] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
W20260812 00:37:59.984292 8201 tablet.cc:2369] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:38:00.029552 8457 tablet.cc:2369] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:38:00.065472 8506 raft_consensus.cc:482] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:38:00.065845 8506 raft_consensus.cc:504] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:38:00.067821 8506 leader_election.cc:288] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911), 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625)
I20260812 00:38:00.076601 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:00.068326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:59411 (local address 127.7.244.66:37911)
0812 00:38:00.068594 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:00.068601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:00.069200 (+ 599us) server_negotiation.cc:407] Connection header received
0812 00:38:00.069437 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:00.069444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:00.069548 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:00.069666 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:38:00.070421 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.071358 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.074231 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.075361 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.075542 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:00.075626 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:00.076110 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 00:38:00.076283 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 00:38:00.076644 8509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:00.068285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:37911 (local address 127.7.244.65:59411)
0812 00:38:00.069048 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:00.069067 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:00.069241 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:00.069870 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:00.069874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:00.069903 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:38:00.070259 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:00.070268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.071507 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.071512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:00.073926 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:00.073957 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.075616 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.075624 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:00.075829 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:00.075871 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 00:38:00.076042 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 00:38:00.076200 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":34,"thread_start_us":440,"threads_started":1}
I20260812 00:38:00.076726 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:00.069619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:51291 (local address 127.7.244.67:34625)
0812 00:38:00.069832 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:00.069838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:00.070071 (+ 233us) server_negotiation.cc:407] Connection header received
0812 00:38:00.070264 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:00.070268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:00.070330 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:00.070418 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:38:00.071015 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.071898 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.074493 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.075604 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.075816 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:00.075910 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:00.076336 (+ 426us) server_negotiation.cc:299] Negotiation successful
0812 00:38:00.076491 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 00:38:00.077247 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:00.069240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:34625 (local address 127.7.244.65:51291)
0812 00:38:00.069961 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:00.069980 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:00.070113 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:00.070460 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:00.070464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:00.070487 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:38:00.070832 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:00.070842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.072055 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.072061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:00.074305 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:00.074319 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.075946 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.075953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:00.076130 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:00.076170 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 00:38:00.076280 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 00:38:00.076429 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":57,"thread_start_us":322,"threads_started":1}
I20260812 00:38:00.078001 8096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e85aba683548b488a3ee3f0dc99b12" candidate_uuid: "da292be6ba9445d180916e1368fe48bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d5c5787a464663aab8ee580d96e0f2" is_pre_election: true
I20260812 00:38:00.078251 8375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e85aba683548b488a3ee3f0dc99b12" candidate_uuid: "da292be6ba9445d180916e1368fe48bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bfba26eb500425a95e74f89672e3f33" is_pre_election: true
I20260812 00:38:00.078626 8096 raft_consensus.cc:2454] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da292be6ba9445d180916e1368fe48bf in term 0.
I20260812 00:38:00.078800 8375 raft_consensus.cc:2454] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da292be6ba9445d180916e1368fe48bf in term 0.
I20260812 00:38:00.079393 7666 leader_election.cc:302] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d5c5787a464663aab8ee580d96e0f2, da292be6ba9445d180916e1368fe48bf; no voters:
I20260812 00:38:00.079818 8506 raft_consensus.cc:2781] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:38:00.079965 8506 raft_consensus.cc:482] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:38:00.080075 8506 raft_consensus.cc:3037] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:38:00.082410 8506 raft_consensus.cc:504] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:38:00.083073 8506 leader_election.cc:288] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 1 election: Requested vote from peers 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911), 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625)
I20260812 00:38:00.083724 8375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e85aba683548b488a3ee3f0dc99b12" candidate_uuid: "da292be6ba9445d180916e1368fe48bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bfba26eb500425a95e74f89672e3f33"
I20260812 00:38:00.083748 8096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e85aba683548b488a3ee3f0dc99b12" candidate_uuid: "da292be6ba9445d180916e1368fe48bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39d5c5787a464663aab8ee580d96e0f2"
I20260812 00:38:00.083931 8375 raft_consensus.cc:3037] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:38:00.083967 8096 raft_consensus.cc:3037] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:38:00.086340 8375 raft_consensus.cc:2454] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da292be6ba9445d180916e1368fe48bf in term 1.
I20260812 00:38:00.086604 8096 raft_consensus.cc:2454] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da292be6ba9445d180916e1368fe48bf in term 1.
I20260812 00:38:00.087015 7666 leader_election.cc:302] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7bfba26eb500425a95e74f89672e3f33, da292be6ba9445d180916e1368fe48bf; no voters:
I20260812 00:38:00.087332 8506 raft_consensus.cc:2781] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:38:00.087838 8506 raft_consensus.cc:686] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 1 LEADER]: Becoming Leader. State: Replica: da292be6ba9445d180916e1368fe48bf, State: Running, Role: LEADER
I20260812 00:38:00.088358 8506 consensus_queue.cc:237] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } }
I20260812 00:38:00.093830 7478 catalog_manager.cc:5495] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf reported cstate change: term changed from 0 to 1, leader changed from <none> to da292be6ba9445d180916e1368fe48bf (127.7.244.65). New cstate: current_term: 1 leader_uuid: "da292be6ba9445d180916e1368fe48bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260812 00:38:00.136664 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:00.130591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.7.244.65:33587)
0812 00:38:00.130791 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:00.130796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:00.130815 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:38:00.131089 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:00.131094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:00.131202 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:00.131366 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:38:00.131760 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.132512 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.134111 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.134567 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.134670 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:00.135334 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:38:00.135942 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:38:00.136083 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:00.136257 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 00:38:00.136349 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 00:38:00.146538 8096 raft_consensus.cc:1261] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 FOLLOWER]: Refusing update from remote peer da292be6ba9445d180916e1368fe48bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:38:00.146538 8375 raft_consensus.cc:1261] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 1 FOLLOWER]: Refusing update from remote peer da292be6ba9445d180916e1368fe48bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:38:00.147722 8515 consensus_queue.cc:1035] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:38:00.148172 8506 consensus_queue.cc:1035] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:38:00.178241 7477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:38:00.178777 7477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:38:00.183660 7477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Sending DeleteTablet for 3 replicas of tablet 44e85aba683548b488a3ee3f0dc99b12
I20260812 00:38:00.185246 7757 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e85aba683548b488a3ee3f0dc99b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:38:00 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:36714
I20260812 00:38:00.185640 8048 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e85aba683548b488a3ee3f0dc99b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:38:00 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:37622
I20260812 00:38:00.185740 8331 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e85aba683548b488a3ee3f0dc99b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:38:00 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:60534
I20260812 00:38:00.186867 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 7631
I20260812 00:38:00.197644 8541 tablet_replica.cc:330] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: stopping tablet replica
I20260812 00:38:00.198487 8541 raft_consensus.cc:2229] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:38:00.199548 8541 raft_consensus.cc:2258] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:38:00.202236 8541 ts_tablet_manager.cc:1905] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:38:00.211004 8541 ts_tablet_manager.cc:1918] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:38:00.211390 8541 log.cc:1196] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/44e85aba683548b488a3ee3f0dc99b12
I20260812 00:38:00.212186 8541 ts_tablet_manager.cc:1939] T 44e85aba683548b488a3ee3f0dc99b12 P da292be6ba9445d180916e1368fe48bf: Deleting consensus metadata
I20260812 00:38:00.214532 7447 catalog_manager.cc:4841] TS da292be6ba9445d180916e1368fe48bf (127.7.244.65:33587): tablet 44e85aba683548b488a3ee3f0dc99b12 (table test-table [id=1ababe758d5e4d07b2f292a899f6002e]) successfully deleted
I20260812 00:38:00.216320 8545 tablet_replica.cc:330] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: stopping tablet replica
I20260812 00:38:00.217170 8544 tablet_replica.cc:330] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: stopping tablet replica
I20260812 00:38:00.217669 8545 raft_consensus.cc:2229] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:38:00.217924 8544 raft_consensus.cc:2229] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:38:00.218339 8545 raft_consensus.cc:2258] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:38:00.218708 8544 raft_consensus.cc:2258] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:38:00.221354 8544 ts_tablet_manager.cc:1905] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:38:00.221342 8545 ts_tablet_manager.cc:1905] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 00:38:00 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:38:00.228878 8544 ts_tablet_manager.cc:1918] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:38:00.229015 8545 ts_tablet_manager.cc:1918] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:38:00.229202 8544 log.cc:1196] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/44e85aba683548b488a3ee3f0dc99b12
I20260812 00:38:00.229310 8545 log.cc:1196] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/44e85aba683548b488a3ee3f0dc99b12
I20260812 00:38:00.229875 8544 ts_tablet_manager.cc:1939] T 44e85aba683548b488a3ee3f0dc99b12 P 7bfba26eb500425a95e74f89672e3f33: Deleting consensus metadata
I20260812 00:38:00.231799 7450 catalog_manager.cc:4841] TS 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625): tablet 44e85aba683548b488a3ee3f0dc99b12 (table test-table [id=1ababe758d5e4d07b2f292a899f6002e]) successfully deleted
I20260812 00:38:00.233945 8545 ts_tablet_manager.cc:1939] T 44e85aba683548b488a3ee3f0dc99b12 P 39d5c5787a464663aab8ee580d96e0f2: Deleting consensus metadata
I20260812 00:38:00.236086 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:00.188307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.7.244.65:33587)
0812 00:38:00.188534 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:00.188541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:00.211325 (+ 22784us) server_negotiation.cc:407] Connection header received
0812 00:38:00.211988 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:00.211996 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:00.212108 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:00.212205 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:00.215298 (+ 3093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.216524 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.219774 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.220480 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.220630 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:00.226695 (+ 6065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:00.226715 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:00.226736 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:00.226797 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:00.231414 (+ 4617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:00.232924 (+ 1510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:00.232932 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:00.232935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:00.233120 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:00.233692 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:00.233702 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:00.233705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:00.234260 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:00.234587 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:00.235377 (+ 790us) server_negotiation.cc:299] Negotiation successful
0812 00:38:00.235682 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 00:38:00.237218 7450 catalog_manager.cc:4841] TS 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911): tablet 44e85aba683548b488a3ee3f0dc99b12 (table test-table [id=1ababe758d5e4d07b2f292a899f6002e]) successfully deleted
I20260812 00:38:00.238070 7909 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:38:00.340934 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 7631
I20260812 00:38:00.373452 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 7925
Aug 12 00:38:00 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:38:00.390604 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:00.374419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.7.244.66:37911)
0812 00:38:00.374630 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:00.374635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:00.374654 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:38:00.374735 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:00.374739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:00.374811 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:00.374926 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:00.376324 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.377133 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.378831 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.379256 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.379359 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:00.384001 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:00.384023 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:00.384042 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:00.384098 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:00.387456 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:00.388336 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:00.388346 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:00.388349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:00.388521 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:00.388956 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:00.388962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:00.388964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:00.389503 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:00.389809 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:00.390046 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 00:38:00.390251 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 00:38:00.391363 8190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:38:00.482913 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 7925
I20260812 00:38:00.514098 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 8209
Aug 12 00:38:00 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:38:00.535611 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:00.514982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.7.244.67:34625)
0812 00:38:00.515160 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:00.515165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:00.515184 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:38:00.515366 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:00.515371 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:00.515438 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:00.515521 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:00.516919 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.517767 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.519334 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:00.519776 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:00.519908 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:00.529387 (+ 9479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:00.529417 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:00.529441 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:00.529504 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:00.532619 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:00.533472 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:00.533480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:00.533483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:00.533647 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:00.534105 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:00.534110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:00.534112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:00.534567 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:00.534808 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:00.535118 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 00:38:00.535312 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 00:38:00.536267 8445 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:38:00.626403 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 8209
I20260812 00:38:00.658674 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 7411
I20260812 00:38:00.659950 7546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:38:00.771232 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 7411
I20260812 00:38:20.803587 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42765
--webserver_interface=127.7.244.126
--webserver_port=39659
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:38:21.193006 8558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:38:21.193406 8558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:38:21.193555 8558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:38:21.201440 8558 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 00:38:21.201572 8558 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 00:38:21.201614 8558 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 00:38:21.201685 8558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:38:21.201772 8558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:38:21.201871 8558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:38:21.201928 8558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:38:21.201984 8558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:38:21.213557 8558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=39659
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:38:21.215037 8558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:38:21.216506 8558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:38:21.242563 8570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:21.242631 8571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:21.242543 8576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:38:21.243564 8558 server_base.cc:1034] running on GCE node
I20260812 00:38:21.244580 8558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:38:21.246439 8558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:38:21.247677 8558 hybrid_clock.cc:648] HybridClock initialized: now 1786495101247603 us; error 45 us; skew 500 ppm
I20260812 00:38:21.248088 8558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:38:21 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495101, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:38:21.252243 8558 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:38:21.254298 8558 webserver.cc:460] Webserver started at http://127.7.244.126:39659/ using document root <none> and password file <none>
I20260812 00:38:21.255198 8558 fs_manager.cc:359] Metadata directory not provided
I20260812 00:38:21.255342 8558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:38:21.262275 8558 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:38:21.265919 8591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:21.267566 8558 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:38:21.267786 8558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "f83e42a0f656413aa42c9fd1921afef5"
format_stamp: "Formatted at 2026-08-12 00:37:57 on dist-test-slave-gjw7"
server_key: "d36733e8f8f3d1231eef6301e7c586c2"
server_key_iv: "7168239da53afccdc522311edecbc222"
server_key_version: "encryptionkey@0"
I20260812 00:38:21.269033 8558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:38:21.292351 8558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:38:21.297371 8558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:38:21.297691 8558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:38:21.317163 8558 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:42765
I20260812 00:38:21.317320 8688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:42765 every 8 connection(s)
I20260812 00:38:21.318950 8558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:38:21.326768 8690 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:38:21.327947 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 8558
I20260812 00:38:21.328832 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:33587
--local_ip_for_outbound_sockets=127.7.244.65
--tserver_master_addrs=127.7.244.126:42765
--webserver_port=34173
--webserver_interface=127.7.244.65
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:38:21.330665 8690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Bootstrap starting.
I20260812 00:38:21.351806 8690 log.cc:824] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Log is configured to *not* fsync() on all Append() calls
I20260812 00:38:21.365809 8690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:38:21.366415 8690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Bootstrap complete.
I20260812 00:38:21.376497 8690 raft_consensus.cc:348] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:38:21.377537 8690 raft_consensus.cc:729] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f83e42a0f656413aa42c9fd1921afef5, State: Initialized, Role: FOLLOWER
I20260812 00:38:21.378273 8690 consensus_queue.cc:260] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:38:21.378533 8690 raft_consensus.cc:388] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:38:21.378662 8690 raft_consensus.cc:482] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:38:21.378839 8690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 1 FOLLOWER]: Advancing to term 2
I20260812 00:38:21.383353 8690 raft_consensus.cc:504] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:38:21.384006 8690 leader_election.cc:302] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f83e42a0f656413aa42c9fd1921afef5; no voters:
I20260812 00:38:21.384720 8690 leader_election.cc:288] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 00:38:21.384867 8695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 00:38:21.386154 8695 raft_consensus.cc:686] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 2 LEADER]: Becoming Leader. State: Replica: f83e42a0f656413aa42c9fd1921afef5, State: Running, Role: LEADER
I20260812 00:38:21.386889 8695 consensus_queue.cc:237] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:38:21.387491 8690 sys_catalog.cc:564] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:38:21.389922 8698 sys_catalog.cc:455] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f83e42a0f656413aa42c9fd1921afef5. Latest consensus state: current_term: 2 leader_uuid: "f83e42a0f656413aa42c9fd1921afef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } } }
I20260812 00:38:21.390374 8698 sys_catalog.cc:458] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:38:21.391649 8696 sys_catalog.cc:455] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f83e42a0f656413aa42c9fd1921afef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } } }
I20260812 00:38:21.392310 8696 sys_catalog.cc:458] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:38:21.396154 8709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:38:21.406325 8709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:38:21.408176 8709 catalog_manager.cc:1259] Loaded cluster ID: 316f0d0dac02472f9be83f8abdfc61cd
I20260812 00:38:21.408329 8709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:38:21.413446 8709 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:38:21.415740 8709 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Loaded TSK: 0
I20260812 00:38:21.418673 8734 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 00:38:21.444316 8709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Generated new TSK 1
I20260812 00:38:21.444772 8709 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 00:38:21.733621 8693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:38:21.733934 8693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:38:21.734053 8693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:38:21.741824 8693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:38:21.741961 8693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:38:21.742106 8693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:38:21.752740 8693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:33587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=34173
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:38:21.754280 8693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:38:21.755722 8693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:38:21.771488 8745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:21.771463 8746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:21.772396 8748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:38:21.773418 8693 server_base.cc:1034] running on GCE node
I20260812 00:38:21.774267 8693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:38:21.775554 8693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:38:21.776827 8693 hybrid_clock.cc:648] HybridClock initialized: now 1786495101776748 us; error 70 us; skew 500 ppm
I20260812 00:38:21.777254 8693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:38:21 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495101, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:38:21.781499 8693 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:38:21.783488 8693 webserver.cc:460] Webserver started at http://127.7.244.65:34173/ using document root <none> and password file <none>
I20260812 00:38:21.784404 8693 fs_manager.cc:359] Metadata directory not provided
I20260812 00:38:21.784564 8693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:38:21.792482 8693 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 00:38:21.796442 8762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:21.798619 8693 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:38:21.798864 8693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "da292be6ba9445d180916e1368fe48bf"
format_stamp: "Formatted at 2026-08-12 00:37:58 on dist-test-slave-gjw7"
server_key: "254ce4dea87055d656ea9752b60f2e20"
server_key_iv: "18c657a391d6f3780dab7650ec63e32c"
server_key_version: "encryptionkey@0"
I20260812 00:38:21.800428 8693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:38:21.817451 8693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:38:21.822111 8693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:38:21.822367 8693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:38:21.823472 8693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:38:21.824975 8693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:38:21.825069 8693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:21.825153 8693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:38:21.825208 8693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:21.858081 8693 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:33587
I20260812 00:38:21.858167 8998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:33587 every 8 connection(s)
I20260812 00:38:21.859788 8693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:38:21.860983 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 8693
I20260812 00:38:21.861881 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:37911
--local_ip_for_outbound_sockets=127.7.244.66
--tserver_master_addrs=127.7.244.126:42765
--webserver_port=33873
--webserver_interface=127.7.244.66
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:38:21 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495101, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:38:21.890285 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:21.863660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:33451)
0812 00:38:21.864630 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:21.864686 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:21.867034 (+ 2348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:21.868772 (+ 1738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:21.868784 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:21.869681 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:38:21.870584 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:21.870619 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:21.872507 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:21.872513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:21.874313 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:21.874333 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:21.875525 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:21.875534 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:21.875836 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:21.876639 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:38:21.876672 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:38:21.879124 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:38:21.884314 (+ 5190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:21.884326 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:21.884347 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:21.885091 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:21.885983 (+ 892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:21.885989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:21.885992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:21.886326 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:21.887559 (+ 1233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:38:21.887569 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:38:21.888220 (+ 651us) client_negotiation.cc:769] Sending connection context
0812 00:38:21.888526 (+ 306us) client_negotiation.cc:240] Negotiation successful
0812 00:38:21.889076 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":371,"threads_started":1}
I20260812 00:38:21.890805 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:21.863402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:33451 (local address 127.7.244.126:42765)
0812 00:38:21.864102 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:21.864123 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:21.864853 (+ 730us) server_negotiation.cc:407] Connection header received
0812 00:38:21.867719 (+ 2866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:21.867733 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:21.868147 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:21.868545 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:21.870890 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:21.872267 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:21.874492 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:21.875218 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:21.875471 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:21.879529 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:21.879544 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:21.879566 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:21.879619 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:21.883950 (+ 4331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:21.885446 (+ 1496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:21.885458 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:21.885467 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:21.885777 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:21.886514 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:21.886521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:21.886525 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:21.887318 (+ 793us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:21.887563 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:21.888877 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0812 00:38:21.889463 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":283,"threads_started":1}
I20260812 00:38:21.894459 8999 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42765
I20260812 00:38:21.894932 8999 heartbeater.cc:464] Registering TS with master...
I20260812 00:38:21.895845 8999 heartbeater.cc:511] Master 127.7.244.126:42765 requested a full tablet report, sending...
I20260812 00:38:21.898693 8627 ts_manager.cc:194] Registered new tserver with Master: da292be6ba9445d180916e1368fe48bf (127.7.244.65:33587)
I20260812 00:38:21.900368 8627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:33451
W20260812 00:38:22.228119 9011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:38:22.228457 9011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:38:22.228586 9011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:38:22.236567 9011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:38:22.236704 9011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:38:22.236866 9011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:38:22.247182 9011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:37911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=33873
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:38:22.248601 9011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:38:22.250073 9011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:38:22.264191 9027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:22.264235 9026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:22.264268 9030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:38:22.265139 9011 server_base.cc:1034] running on GCE node
I20260812 00:38:22.265839 9011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:38:22.266871 9011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:38:22.268085 9011 hybrid_clock.cc:648] HybridClock initialized: now 1786495102268037 us; error 46 us; skew 500 ppm
I20260812 00:38:22.268414 9011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:38:22 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495102, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:38:22.276053 9011 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:38:22.277778 9011 webserver.cc:460] Webserver started at http://127.7.244.66:33873/ using document root <none> and password file <none>
I20260812 00:38:22.278501 9011 fs_manager.cc:359] Metadata directory not provided
I20260812 00:38:22.278635 9011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:38:22.284610 9011 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 00:38:22.287875 9040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:22.289502 9011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:38:22.289690 9011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "39d5c5787a464663aab8ee580d96e0f2"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "21b08c193b95a4b2bd0269714d558298"
server_key_iv: "1a86e1c86d70f62b7d56cf297b069762"
server_key_version: "encryptionkey@0"
I20260812 00:38:22.290759 9011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:38:22.318121 9011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:38:22.322736 9011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:38:22.323046 9011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:38:22.324144 9011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:38:22.325610 9011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:38:22.325711 9011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:22.325821 9011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:38:22.325898 9011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:22.359198 9011 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:37911
I20260812 00:38:22.359278 9265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:37911 every 8 connection(s)
I20260812 00:38:22.360754 9011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:38:22.368820 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 9011
I20260812 00:38:22.369489 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:34625
--local_ip_for_outbound_sockets=127.7.244.67
--tserver_master_addrs=127.7.244.126:42765
--webserver_port=42033
--webserver_interface=127.7.244.67
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 00:38:22 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495102, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:38:22.383558 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.364081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:49447 (local address 127.7.244.126:42765)
0812 00:38:22.364360 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:22.364368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:22.365778 (+ 1410us) server_negotiation.cc:407] Connection header received
0812 00:38:22.367695 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:22.367701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:22.367826 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:22.367928 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:22.369786 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.370527 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.372209 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.372661 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.372782 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.376969 (+ 4187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:22.376975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:22.376979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:22.377019 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:22.379734 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:22.380839 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:22.380843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:22.380845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:22.380991 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:22.381531 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:22.381537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:22.381539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:22.381949 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:22.382102 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:22.382916 (+ 814us) server_negotiation.cc:299] Negotiation successful
0812 00:38:22.383208 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260812 00:38:22.384546 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.364719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:49447)
0812 00:38:22.365580 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:22.365630 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:22.367365 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:22.368152 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:22.368161 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:22.368782 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:38:22.369511 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.369536 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.370717 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.370721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.372032 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.372049 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.372919 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.372928 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.373235 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.373994 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:38:22.374020 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:38:22.376623 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:38:22.379933 (+ 3310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:22.379946 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:22.379969 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:22.380682 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:22.381105 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:22.381110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:22.381113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:22.381395 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:22.382108 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:38:22.382115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:38:22.382506 (+ 391us) client_negotiation.cc:769] Sending connection context
0812 00:38:22.382771 (+ 265us) client_negotiation.cc:240] Negotiation successful
0812 00:38:22.383250 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":301,"threads_started":1}
I20260812 00:38:22.386380 9268 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42765
I20260812 00:38:22.386756 9268 heartbeater.cc:464] Registering TS with master...
I20260812 00:38:22.387951 9268 heartbeater.cc:511] Master 127.7.244.126:42765 requested a full tablet report, sending...
I20260812 00:38:22.389766 8627 ts_manager.cc:194] Registered new tserver with Master: 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911)
I20260812 00:38:22.390635 8627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:49447
W20260812 00:38:22.735379 9272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:38:22.735725 9272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:38:22.735867 9272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:38:22.743376 9272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:38:22.743525 9272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:38:22.743688 9272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:38:22.754009 9272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:34625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=42033
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:38:22.755438 9272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:38:22.756861 9272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:38:22.771061 9281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:22.771036 9287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:22.771595 9295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:38:22.771878 9272 server_base.cc:1034] running on GCE node
I20260812 00:38:22.772634 9272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:38:22.773850 9272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:38:22.775098 9272 hybrid_clock.cc:648] HybridClock initialized: now 1786495102775014 us; error 67 us; skew 500 ppm
I20260812 00:38:22.775485 9272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:38:22 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495102, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:38:22.779022 9272 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:38:22.780790 9272 webserver.cc:460] Webserver started at http://127.7.244.67:42033/ using document root <none> and password file <none>
I20260812 00:38:22.781644 9272 fs_manager.cc:359] Metadata directory not provided
I20260812 00:38:22.781807 9272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:38:22.788519 9272 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 00:38:22.792008 9310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:22.793723 9272 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260812 00:38:22.793933 9272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "7bfba26eb500425a95e74f89672e3f33"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "e0f3bc8b28139f04bcb00d9956db082b"
server_key_iv: "06b7400af251f8d900e6cda513a53e01"
server_key_version: "encryptionkey@0"
I20260812 00:38:22.795209 9272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:38:22.813805 9272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:38:22.818495 9272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:38:22.818742 9272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:38:22.819802 9272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:38:22.821269 9272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:38:22.821390 9272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260812 00:38:22.821501 9272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:38:22.821584 9272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:22.855011 9272 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:34625
I20260812 00:38:22.855083 9533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:34625 every 8 connection(s)
I20260812 00:38:22.856734 9272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:38:22.862762 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 9272
Aug 12 00:38:22 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495102, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:38:22.880446 9544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.863912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39032 (local address 127.7.244.126:42765)
0812 00:38:22.864553 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:22.864559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:22.864923 (+ 364us) server_negotiation.cc:407] Connection header received
0812 00:38:22.865074 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:22.865080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:22.865157 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:22.865274 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:22.867070 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.868131 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.869792 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.870240 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.870357 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.873572 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:22.873579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:22.873622 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:22.873669 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:22.877276 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:22.878152 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:22.878156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:22.878158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:22.878300 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:22.878770 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:22.878775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:22.878777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:22.879117 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:22.879379 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:22.879758 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 00:38:22.880014 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":400,"threads_started":1}
I20260812 00:38:22.880782 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.859671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:42689 (local address 127.7.244.126:42765)
0812 00:38:22.859907 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:22.859913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:22.861377 (+ 1464us) server_negotiation.cc:407] Connection header received
0812 00:38:22.863882 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:22.863888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:22.863981 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:22.864079 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:22.866109 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.866844 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.869014 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.869498 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.869613 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.873690 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:22.873697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:22.873705 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:22.873757 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:22.877120 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:22.878100 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:22.878105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:22.878108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:22.878266 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:22.878870 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:22.878874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:22.878876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:22.879167 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:22.879312 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:22.880128 (+ 816us) server_negotiation.cc:299] Negotiation successful
0812 00:38:22.880369 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 00:38:22.882190 9543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.860241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:42689)
0812 00:38:22.861145 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:22.861196 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:22.863508 (+ 2312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:22.864399 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:22.864412 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:22.865151 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:38:22.865928 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.865951 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.867029 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.867035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.868857 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.868874 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.869752 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.869760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.870039 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.870748 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:38:22.870774 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:38:22.873426 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:38:22.877375 (+ 3949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:22.877384 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:22.877403 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:22.877945 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:22.878446 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:22.878452 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:22.878455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:22.878729 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:22.879314 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:38:22.879324 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:38:22.879727 (+ 403us) client_negotiation.cc:769] Sending connection context
0812 00:38:22.880003 (+ 276us) client_negotiation.cc:240] Negotiation successful
0812 00:38:22.880643 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":361,"threads_started":1}
I20260812 00:38:22.883968 9536 heartbeater.cc:347] Connected to a master server at 127.7.244.126:42765
I20260812 00:38:22.884337 9536 heartbeater.cc:464] Registering TS with master...
I20260812 00:38:22.885167 9536 heartbeater.cc:511] Master 127.7.244.126:42765 requested a full tablet report, sending...
I20260812 00:38:22.886960 8625 ts_manager.cc:194] Registered new tserver with Master: 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625)
I20260812 00:38:22.887926 8625 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:42689
I20260812 00:38:22.897382 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:38:22.904157 8999 heartbeater.cc:503] Master 127.7.244.126:42765 was elected leader, sending a full tablet report...
I20260812 00:38:22.905558 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.898663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.7.244.126:42765)
0812 00:38:22.898854 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:22.898859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:22.899246 (+ 387us) server_negotiation.cc:407] Connection header received
0812 00:38:22.899454 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:22.899458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:22.899635 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:22.899794 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:38:22.900313 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.901109 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.902983 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.903516 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.903642 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.904422 (+ 780us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:38:22.904782 (+ 360us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 00:38:22.905234 (+ 452us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39040: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 00:38:22.905962 9007 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39040: token expired
W20260812 00:38:22.906352 8475 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.244.126:42765)
I20260812 00:38:22.921145 9007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.907000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39056 (local address 127.7.244.126:42765)
0812 00:38:22.907189 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:22.907195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:22.907241 (+ 46us) server_negotiation.cc:407] Connection header received
0812 00:38:22.907433 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:22.907439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:22.907522 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:22.907609 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:22.909166 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.910027 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.911541 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.911982 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.912105 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.915294 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:22.915300 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:22.915303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:22.915347 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:22.918088 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:22.918959 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:22.918965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:22.918967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:22.919116 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:22.919544 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:22.919548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:22.919550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:22.919925 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:22.920167 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:22.920573 (+ 406us) server_negotiation.cc:299] Negotiation successful
0812 00:38:22.920824 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 00:38:22.925156 8625 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:38:22.932860 8625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:38:22.969578 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.946958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.7.244.65:33587)
0812 00:38:22.960331 (+ 13373us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:22.960343 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:22.960395 (+ 52us) server_negotiation.cc:407] Connection header received
0812 00:38:22.960519 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:22.960528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:22.960857 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:22.961808 (+ 951us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:38:22.962084 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.963641 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.966554 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.968074 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.968245 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.968524 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:22.968988 (+ 464us) server_negotiation.cc:299] Negotiation successful
0812 00:38:22.969194 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13188,"thread_start_us":13033,"threads_started":1}
I20260812 00:38:22.969605 9557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.946655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:38974)
0812 00:38:22.948029 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:22.948080 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:22.948270 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:22.961185 (+ 12915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:22.961198 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:22.961264 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:38:22.961863 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.961876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.963818 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.963823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.966353 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.966373 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.968355 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.968362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.968588 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.968659 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 00:38:22.968818 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 00:38:22.969080 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1085,"mutex_wait_us":58,"thread_start_us":12891,"threads_started":1}
I20260812 00:38:22.969964 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.947603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.7.244.67:34625)
0812 00:38:22.960782 (+ 13179us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:22.960791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:22.960827 (+ 36us) server_negotiation.cc:407] Connection header received
0812 00:38:22.960936 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:22.960943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:22.961186 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:22.961454 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:38:22.962346 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.963870 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.966998 (+ 3128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.968467 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.968609 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.968849 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:22.969426 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 00:38:22.969606 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12982,"thread_start_us":12815,"threads_started":1}
I20260812 00:38:22.970420 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.948865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38360 (local address 127.7.244.66:37911)
0812 00:38:22.961065 (+ 12200us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:22.961074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:22.961923 (+ 849us) server_negotiation.cc:407] Connection header received
0812 00:38:22.962032 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:22.962038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:22.962207 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:22.962406 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:38:22.962968 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.965071 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.967520 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.968706 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.968847 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.969033 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:22.969914 (+ 881us) server_negotiation.cc:299] Negotiation successful
0812 00:38:22.970112 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12035,"thread_start_us":11906,"threads_started":1}
I20260812 00:38:22.970448 9555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.947140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:34625 (local address 127.0.0.1:36344)
0812 00:38:22.959927 (+ 12787us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:22.959954 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:22.960128 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:22.961594 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:22.961598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:22.961625 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:38:22.962105 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.962118 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.964023 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.964028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.966793 (+ 2765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.966811 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.968700 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.968708 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.968924 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.968959 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:38:22.969050 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 00:38:22.969193 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12602,"mutex_wait_us":345,"spinlock_wait_cycles":21248,"thread_start_us":433,"threads_started":1}
I20260812 00:38:22.970949 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:22.948477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:37911 (local address 127.0.0.1:38360)
0812 00:38:22.961581 (+ 13104us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:22.961602 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:22.961752 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:22.962407 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:22.962411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:22.962440 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:38:22.962786 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.962801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.965251 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.965257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.967337 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:22.967349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:22.969192 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:22.969199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:22.969427 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:22.969462 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:38:22.969561 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:38:22.969693 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12971,"mutex_wait_us":3,"spinlock_wait_cycles":896,"thread_start_us":12821,"threads_started":1}
I20260812 00:38:22.974709 9145 tablet_service.cc:1467] Processing CreateTablet for tablet 972ec7063b2b49d686a6350fdb5c8391 (DEFAULT_TABLE table=test-table [id=6913dd98f1804a8ba42ed4a878986d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:38:22.974720 8865 tablet_service.cc:1467] Processing CreateTablet for tablet 972ec7063b2b49d686a6350fdb5c8391 (DEFAULT_TABLE table=test-table [id=6913dd98f1804a8ba42ed4a878986d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:38:22.974720 9402 tablet_service.cc:1467] Processing CreateTablet for tablet 972ec7063b2b49d686a6350fdb5c8391 (DEFAULT_TABLE table=test-table [id=6913dd98f1804a8ba42ed4a878986d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:38:22.976205 9145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972ec7063b2b49d686a6350fdb5c8391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:38:22.976315 9402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972ec7063b2b49d686a6350fdb5c8391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:38:22.976332 8865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972ec7063b2b49d686a6350fdb5c8391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:38:22.990062 9562 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: Bootstrap starting.
I20260812 00:38:22.994000 9562 tablet_bootstrap.cc:654] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: Neither blocks nor log segments found. Creating new log.
I20260812 00:38:22.994155 9563 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Bootstrap starting.
I20260812 00:38:22.995896 9562 log.cc:824] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: Log is configured to *not* fsync() on all Append() calls
I20260812 00:38:22.996630 9564 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Bootstrap starting.
I20260812 00:38:22.998433 9563 tablet_bootstrap.cc:654] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Neither blocks nor log segments found. Creating new log.
I20260812 00:38:22.999979 9563 log.cc:824] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Log is configured to *not* fsync() on all Append() calls
I20260812 00:38:23.000797 9564 tablet_bootstrap.cc:654] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Neither blocks nor log segments found. Creating new log.
I20260812 00:38:23.001081 9562 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: No bootstrap required, opened a new log
I20260812 00:38:23.001449 9562 ts_tablet_manager.cc:1397] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260812 00:38:23.002218 9564 log.cc:824] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Log is configured to *not* fsync() on all Append() calls
I20260812 00:38:23.003122 9563 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: No bootstrap required, opened a new log
I20260812 00:38:23.003484 9563 ts_tablet_manager.cc:1397] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 00:38:23.006297 9564 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: No bootstrap required, opened a new log
I20260812 00:38:23.006688 9564 ts_tablet_manager.cc:1397] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260812 00:38:23.011632 9563 raft_consensus.cc:348] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.012257 9563 raft_consensus.cc:374] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:38:23.012439 9563 raft_consensus.cc:729] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da292be6ba9445d180916e1368fe48bf, State: Initialized, Role: FOLLOWER
I20260812 00:38:23.012498 9562 raft_consensus.cc:348] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.013077 9562 raft_consensus.cc:374] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:38:23.013248 9562 raft_consensus.cc:729] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d5c5787a464663aab8ee580d96e0f2, State: Initialized, Role: FOLLOWER
I20260812 00:38:23.013250 9563 consensus_queue.cc:260] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.014122 9562 consensus_queue.cc:260] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.015538 9563 ts_tablet_manager.cc:1428] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260812 00:38:23.017764 9268 heartbeater.cc:503] Master 127.7.244.126:42765 was elected leader, sending a full tablet report...
I20260812 00:38:23.017725 9562 ts_tablet_manager.cc:1428] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260812 00:38:23.018411 9564 raft_consensus.cc:348] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.018781 9564 raft_consensus.cc:374] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:38:23.018843 9564 raft_consensus.cc:729] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bfba26eb500425a95e74f89672e3f33, State: Initialized, Role: FOLLOWER
I20260812 00:38:23.019618 9564 consensus_queue.cc:260] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.021720 9536 heartbeater.cc:503] Master 127.7.244.126:42765 was elected leader, sending a full tablet report...
I20260812 00:38:23.022357 9564 ts_tablet_manager.cc:1428] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260812 00:38:23.028270 9571 raft_consensus.cc:482] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:38:23.028564 9571 raft_consensus.cc:504] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.030578 9571 leader_election.cc:288] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da292be6ba9445d180916e1368fe48bf (127.7.244.65:33587), 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625)
I20260812 00:38:23.037374 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.031642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:46333 (local address 127.7.244.65:33587)
0812 00:38:23.031851 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:23.031857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:23.031875 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:38:23.031949 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:23.031954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:23.032027 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:23.032121 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:38:23.032859 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.033697 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.035582 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.036401 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.036580 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:23.036664 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:23.036961 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 00:38:23.037096 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:38:23.037576 9578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.030818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.7.244.66:46333)
0812 00:38:23.031612 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.031630 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:23.031778 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:23.032180 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:23.032187 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:23.032225 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:38:23.032692 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:23.032705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.033843 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.033849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:23.035385 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:23.035400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.036646 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.036653 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:23.036793 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:23.036819 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 00:38:23.036908 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 00:38:23.037057 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":4,"thread_start_us":830,"threads_started":1}
I20260812 00:38:23.038115 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.031862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:40783 (local address 127.7.244.67:34625)
0812 00:38:23.032062 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:23.032068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:23.032796 (+ 728us) server_negotiation.cc:407] Connection header received
0812 00:38:23.032958 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:23.032963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:23.033023 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:23.033101 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:38:23.033731 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.034449 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.036224 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.037058 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.037214 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:23.037317 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:23.037726 (+ 409us) server_negotiation.cc:299] Negotiation successful
0812 00:38:23.037872 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:38:23.038713 9581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.031344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:34625 (local address 127.7.244.66:40783)
0812 00:38:23.032684 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.032700 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:23.032824 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:23.033132 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:23.033137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:23.033163 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:38:23.033577 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:23.033588 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.034593 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.034599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:23.036057 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:23.036071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.037317 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.037325 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:23.037518 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:23.037559 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 00:38:23.038352 (+ 793us) client_negotiation.cc:240] Negotiation successful
0812 00:38:23.038471 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"thread_start_us":991,"threads_started":1}
I20260812 00:38:23.038663 8914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972ec7063b2b49d686a6350fdb5c8391" candidate_uuid: "39d5c5787a464663aab8ee580d96e0f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da292be6ba9445d180916e1368fe48bf" is_pre_election: true
I20260812 00:38:23.039268 8914 raft_consensus.cc:2454] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39d5c5787a464663aab8ee580d96e0f2 in term 0.
I20260812 00:38:23.039783 9448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972ec7063b2b49d686a6350fdb5c8391" candidate_uuid: "39d5c5787a464663aab8ee580d96e0f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bfba26eb500425a95e74f89672e3f33" is_pre_election: true
I20260812 00:38:23.040159 9044 leader_election.cc:302] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d5c5787a464663aab8ee580d96e0f2, da292be6ba9445d180916e1368fe48bf; no voters:
I20260812 00:38:23.040271 9448 raft_consensus.cc:2454] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39d5c5787a464663aab8ee580d96e0f2 in term 0.
I20260812 00:38:23.040550 9571 raft_consensus.cc:2781] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:38:23.040714 9571 raft_consensus.cc:482] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:38:23.040835 9571 raft_consensus.cc:3037] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:38:23.043179 9571 raft_consensus.cc:504] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.043891 9571 leader_election.cc:288] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [CANDIDATE]: Term 1 election: Requested vote from peers da292be6ba9445d180916e1368fe48bf (127.7.244.65:33587), 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625)
I20260812 00:38:23.044510 8914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972ec7063b2b49d686a6350fdb5c8391" candidate_uuid: "39d5c5787a464663aab8ee580d96e0f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da292be6ba9445d180916e1368fe48bf"
I20260812 00:38:23.044627 9448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "972ec7063b2b49d686a6350fdb5c8391" candidate_uuid: "39d5c5787a464663aab8ee580d96e0f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bfba26eb500425a95e74f89672e3f33"
I20260812 00:38:23.044764 8914 raft_consensus.cc:3037] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:38:23.044847 9448 raft_consensus.cc:3037] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:38:23.047302 9448 raft_consensus.cc:2454] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39d5c5787a464663aab8ee580d96e0f2 in term 1.
I20260812 00:38:23.047406 8914 raft_consensus.cc:2454] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39d5c5787a464663aab8ee580d96e0f2 in term 1.
I20260812 00:38:23.047892 9048 leader_election.cc:302] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39d5c5787a464663aab8ee580d96e0f2, 7bfba26eb500425a95e74f89672e3f33; no voters:
I20260812 00:38:23.048293 9571 raft_consensus.cc:2781] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:38:23.049068 9571 raft_consensus.cc:686] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 LEADER]: Becoming Leader. State: Replica: 39d5c5787a464663aab8ee580d96e0f2, State: Running, Role: LEADER
I20260812 00:38:23.049757 9571 consensus_queue.cc:237] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:23.055058 8627 catalog_manager.cc:5495] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66). New cstate: current_term: 1 leader_uuid: "39d5c5787a464663aab8ee580d96e0f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } health_report { overall_health: HEALTHY } } }
I20260812 00:38:23.088413 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.082265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.7.244.66:37911)
0812 00:38:23.082517 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:23.082522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:23.082610 (+ 88us) server_negotiation.cc:407] Connection header received
0812 00:38:23.082836 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:23.082841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:23.082965 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:23.083138 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:38:23.083584 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.084368 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.085931 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.086416 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.086534 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:23.087113 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:38:23.087767 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:38:23.087894 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:23.087988 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 00:38:23.088070 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 00:38:23.098206 9448 raft_consensus.cc:1261] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 1 FOLLOWER]: Refusing update from remote peer 39d5c5787a464663aab8ee580d96e0f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:38:23.098201 8914 raft_consensus.cc:1261] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Refusing update from remote peer 39d5c5787a464663aab8ee580d96e0f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:38:23.099293 9571 consensus_queue.cc:1035] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:38:23.099638 9583 consensus_queue.cc:1035] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 00:38:23.108688 9537 tablet.cc:2369] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:38:23.112435 9005 tablet.cc:2369] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:38:23.112541 9587 mvcc.cc:205] Tried to move back new op lower bound from 7317483942277652480 to 7317483942096265216. Current Snapshot: MvccSnapshot[applied={T|T < 7317483942277652480}]
I20260812 00:38:23.115831 9588 mvcc.cc:205] Tried to move back new op lower bound from 7317483942277652480 to 7317483942096265216. Current Snapshot: MvccSnapshot[applied={T|T < 7317483942277652480}]
I20260812 00:38:23.133968 8627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:38:23.134315 8627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:38:23.138628 8627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Sending DeleteTablet for 3 replicas of tablet 972ec7063b2b49d686a6350fdb5c8391
I20260812 00:38:23.140223 9402 tablet_service.cc:1514] Processing DeleteTablet for tablet 972ec7063b2b49d686a6350fdb5c8391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:38:23 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:36344
I20260812 00:38:23.140347 8865 tablet_service.cc:1514] Processing DeleteTablet for tablet 972ec7063b2b49d686a6350fdb5c8391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:38:23 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:38974
I20260812 00:38:23.140379 9145 tablet_service.cc:1514] Processing DeleteTablet for tablet 972ec7063b2b49d686a6350fdb5c8391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:38:23 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:38360
I20260812 00:38:23.140522 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 8693
I20260812 00:38:23.162555 9556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.141947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38984 (local address 127.7.244.65:33587)
0812 00:38:23.142155 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:23.142162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:23.142184 (+ 22us) server_negotiation.cc:407] Connection header received
0812 00:38:23.142253 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:23.142259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:23.142361 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:23.142463 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:23.144592 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.145979 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.148280 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.149095 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.149244 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:23.153769 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:23.153798 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:23.153824 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:23.153891 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:23.158687 (+ 4796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:23.159784 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:23.159797 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:23.159802 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:23.160036 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:23.160509 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:23.160516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:23.160519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:23.161132 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:23.161509 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:23.161909 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 00:38:23.162159 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 00:38:23.163388 8990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:38:23.165580 9608 tablet_replica.cc:330] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: stopping tablet replica
I20260812 00:38:23.166188 9608 raft_consensus.cc:2229] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:38:23.167062 9608 raft_consensus.cc:2258] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:38:23.169817 9608 ts_tablet_manager.cc:1905] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:38:23.182900 9608 ts_tablet_manager.cc:1918] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:38:23.183145 9608 log.cc:1196] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/972ec7063b2b49d686a6350fdb5c8391
I20260812 00:38:23.183521 9611 tablet_replica.cc:330] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: stopping tablet replica
I20260812 00:38:23.183854 9608 ts_tablet_manager.cc:1939] T 972ec7063b2b49d686a6350fdb5c8391 P 7bfba26eb500425a95e74f89672e3f33: Deleting consensus metadata
I20260812 00:38:23.186645 9611 raft_consensus.cc:2229] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:38:23.187321 8602 catalog_manager.cc:4841] TS 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625): tablet 972ec7063b2b49d686a6350fdb5c8391 (table test-table [id=6913dd98f1804a8ba42ed4a878986d5c]) successfully deleted
I20260812 00:38:23.187765 9611 raft_consensus.cc:2258] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:38:23.190670 9611 ts_tablet_manager.cc:1905] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:38:23.200950 9611 ts_tablet_manager.cc:1918] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:38:23.201205 9611 log.cc:1196] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/972ec7063b2b49d686a6350fdb5c8391
I20260812 00:38:23.201894 9611 ts_tablet_manager.cc:1939] T 972ec7063b2b49d686a6350fdb5c8391 P 39d5c5787a464663aab8ee580d96e0f2: Deleting consensus metadata
I20260812 00:38:23.204198 8602 catalog_manager.cc:4841] TS 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911): tablet 972ec7063b2b49d686a6350fdb5c8391 (table test-table [id=6913dd98f1804a8ba42ed4a878986d5c]) successfully deleted
I20260812 00:38:23.279495 9606 tablet_replica.cc:330] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: stopping tablet replica
I20260812 00:38:23.279749 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 8693
W20260812 00:38:23.322696 8596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.244.65:33587 (error 108)
I20260812 00:38:23.322976 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 9011
I20260812 00:38:23.339443 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.338886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:38990)
0812 00:38:23.339102 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.339246 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 00:38:23.339759 8596 catalog_manager.cc:4584] TS da292be6ba9445d180916e1368fe48bf (127.7.244.65:33587): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 972ec7063b2b49d686a6350fdb5c8391: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
I20260812 00:38:23.343700 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.324637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38384 (local address 127.7.244.66:37911)
0812 00:38:23.324826 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:23.324833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:23.324852 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:38:23.324917 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:23.324923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:23.325008 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:23.325108 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:23.327261 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.328541 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.330676 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.331391 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.331527 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:23.335544 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:23.335558 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:23.335579 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:23.335642 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:23.340467 (+ 4825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:23.341327 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:23.341338 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:23.341342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:23.341556 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:23.341973 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:23.341979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:23.341982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:23.342446 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:23.342788 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:23.343043 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 00:38:23.343297 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:38:23.344587 9257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:38:23.393759 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.393101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:39000)
0812 00:38:23.393414 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.393565 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260812 00:38:23.395076 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.394610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:39002)
0812 00:38:23.394803 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.394924 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 00:38:23.455799 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.455164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:39018)
0812 00:38:23.455450 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.455575 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260812 00:38:23.457003 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.456591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:39028)
0812 00:38:23.456764 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.456858 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 00:38:23.460354 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 9011
I20260812 00:38:23.493845 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 9272
I20260812 00:38:23.510167 9559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.494683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.7.244.67:34625)
0812 00:38:23.494899 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:23.494904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:23.494963 (+ 59us) server_negotiation.cc:407] Connection header received
0812 00:38:23.495130 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:23.495134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:23.495200 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:23.495279 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:23.496869 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.497801 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.499346 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:23.499815 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:23.499949 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:23.503139 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:23.503161 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:23.503187 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:23.503243 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:23.506368 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:23.507288 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:23.507300 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:23.507303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:23.507604 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:23.508084 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:23.508088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:23.508090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:23.508609 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:23.508940 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:23.509539 (+ 599us) server_negotiation.cc:299] Negotiation successful
0812 00:38:23.509811 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 00:38:23.511068 9523 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:38:23.538627 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.537975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:39036)
0812 00:38:23.538285 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.538414 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260812 00:38:23.540012 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.539463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:39050)
0812 00:38:23.539733 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.539843 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260812 00:38:23.601495 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 9272
I20260812 00:38:23.625237 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 8558
I20260812 00:38:23.626752 8682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:38:23.742111 9560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:23.741489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:33587 (local address 127.0.0.1:39064)
0812 00:38:23.741735 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:23.741904 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:33587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 00:38:23.742419 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 8558
I20260812 00:38:43.781919 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42765
--webserver_interface=127.7.244.126
--webserver_port=39659
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:38:44.162338 9625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:38:44.162683 9625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:38:44.162833 9625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:38:44.170472 9625 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 00:38:44.170612 9625 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 00:38:44.170667 9625 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 00:38:44.170748 9625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:38:44.170825 9625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:38:44.170928 9625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:38:44.170996 9625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:38:44.171075 9625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:38:44.181735 9625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=39659
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:38:44.183220 9625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:38:44.184705 9625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:38:44.198411 9637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:44.198378 9638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:44.198444 9644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:38:44.199163 9625 server_base.cc:1034] running on GCE node
I20260812 00:38:44.200213 9625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:38:44.201920 9625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:38:44.203178 9625 hybrid_clock.cc:648] HybridClock initialized: now 1786495124203095 us; error 63 us; skew 500 ppm
I20260812 00:38:44.203577 9625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:38:44 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:38:44.211853 9625 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:38:44.213704 9625 webserver.cc:460] Webserver started at http://127.7.244.126:39659/ using document root <none> and password file <none>
I20260812 00:38:44.214597 9625 fs_manager.cc:359] Metadata directory not provided
I20260812 00:38:44.214708 9625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:38:44.221081 9625 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 00:38:44.224634 9654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:44.226274 9625 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:38:44.226464 9625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "f83e42a0f656413aa42c9fd1921afef5"
format_stamp: "Formatted at 2026-08-12 00:37:57 on dist-test-slave-gjw7"
server_key: "d36733e8f8f3d1231eef6301e7c586c2"
server_key_iv: "7168239da53afccdc522311edecbc222"
server_key_version: "encryptionkey@0"
I20260812 00:38:44.227581 9625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:38:44.241990 9625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:38:44.247306 9625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:38:44.247632 9625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:38:44.267347 9625 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:42765
I20260812 00:38:44.267369 9747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:42765 every 8 connection(s)
I20260812 00:38:44.268994 9625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:38:44.275278 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 9625
I20260812 00:38:44.276199 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:33587
--local_ip_for_outbound_sockets=127.7.244.65
--tserver_master_addrs=127.7.244.126:42765
--webserver_port=34173
--webserver_interface=127.7.244.65
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:38:44.276819 9751 sys_catalog.cc:263] Verifying existing consensus state
I20260812 00:38:44.280539 9751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Bootstrap starting.
I20260812 00:38:44.306551 9751 log.cc:824] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Log is configured to *not* fsync() on all Append() calls
I20260812 00:38:44.329672 9751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:38:44.330430 9751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Bootstrap complete.
I20260812 00:38:44.344674 9751 raft_consensus.cc:348] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:38:44.346109 9751 raft_consensus.cc:729] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f83e42a0f656413aa42c9fd1921afef5, State: Initialized, Role: FOLLOWER
I20260812 00:38:44.347021 9751 consensus_queue.cc:260] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:38:44.347332 9751 raft_consensus.cc:388] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:38:44.347472 9751 raft_consensus.cc:482] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:38:44.347656 9751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 2 FOLLOWER]: Advancing to term 3
I20260812 00:38:44.353739 9751 raft_consensus.cc:504] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:38:44.354557 9751 leader_election.cc:302] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f83e42a0f656413aa42c9fd1921afef5; no voters:
I20260812 00:38:44.355414 9751 leader_election.cc:288] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 00:38:44.355566 9758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 00:38:44.356860 9758 raft_consensus.cc:686] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [term 3 LEADER]: Becoming Leader. State: Replica: f83e42a0f656413aa42c9fd1921afef5, State: Running, Role: LEADER
I20260812 00:38:44.357621 9758 consensus_queue.cc:237] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } }
I20260812 00:38:44.358165 9751 sys_catalog.cc:564] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:38:44.360435 9762 sys_catalog.cc:455] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f83e42a0f656413aa42c9fd1921afef5. Latest consensus state: current_term: 3 leader_uuid: "f83e42a0f656413aa42c9fd1921afef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } } }
I20260812 00:38:44.360314 9759 sys_catalog.cc:455] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f83e42a0f656413aa42c9fd1921afef5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f83e42a0f656413aa42c9fd1921afef5" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42765 } } }
I20260812 00:38:44.360834 9762 sys_catalog.cc:458] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:38:44.360837 9759 sys_catalog.cc:458] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5 [sys.catalog]: This master's current role is: LEADER
I20260812 00:38:44.374502 9771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:38:44.386447 9771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:38:44.388409 9771 catalog_manager.cc:1259] Loaded cluster ID: 316f0d0dac02472f9be83f8abdfc61cd
I20260812 00:38:44.388623 9771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:38:44.393890 9771 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:38:44.396117 9771 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Loaded TSK: 0
I20260812 00:38:44.396425 9771 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f83e42a0f656413aa42c9fd1921afef5: Loaded TSK: 1
F20260812 00:38:44.397853 9771 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa886d029ad google::LogMessage::Fail() at ??:0
@ 0x7fa886d0588a google::LogMessage::SendToLog() at ??:0
@ 0x7fa886d024de google::LogMessage::Flush() at ??:0
@ 0x7fa886d03a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa88aa667d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa88aa5573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa89c7351c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa89c734783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa89c7c02a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fa89c7c00c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fa89c80a5e9 std::function<>::operator()() at ??:0
@ 0x7fa89c738cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa89c736c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa89c7be610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa89c7be439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa88994e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa88995191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa889951799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa88992ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa88cc066db start_thread at ??:0
@ 0x7fa885b9c61f clone at ??:0
W20260812 00:38:44.745458 9756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:38:44.745801 9756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:38:44.745960 9756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:38:44.753918 9756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:38:44.754063 9756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:38:44.754274 9756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:38:44.765270 9756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:33587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=34173
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:38:44.766831 9756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:38:44.768342 9756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:38:44.783321 9864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:44.783435 9860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:44.783403 9861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:38:44.784735 9756 server_base.cc:1034] running on GCE node
I20260812 00:38:44.785578 9756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:38:44.786758 9756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:38:44.788028 9756 hybrid_clock.cc:648] HybridClock initialized: now 1786495124787976 us; error 45 us; skew 500 ppm
I20260812 00:38:44.788435 9756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:38:44 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:38:44.791953 9756 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:38:44.793746 9756 webserver.cc:460] Webserver started at http://127.7.244.65:34173/ using document root <none> and password file <none>
I20260812 00:38:44.794598 9756 fs_manager.cc:359] Metadata directory not provided
I20260812 00:38:44.794762 9756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:38:44.800700 9756 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 00:38:44.803997 9877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:44.805583 9756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 00:38:44.805768 9756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "da292be6ba9445d180916e1368fe48bf"
format_stamp: "Formatted at 2026-08-12 00:37:58 on dist-test-slave-gjw7"
server_key: "254ce4dea87055d656ea9752b60f2e20"
server_key_iv: "18c657a391d6f3780dab7650ec63e32c"
server_key_version: "encryptionkey@0"
I20260812 00:38:44.806828 9756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:38:44.824493 9756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:38:44.828881 9756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:38:44.829169 9756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:38:44.830276 9756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:38:44.832337 9898 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 00:38:44.838743 9756 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 00:38:44.838919 9756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260812 00:38:44.839062 9756 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 00:38:44.842692 9756 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 00:38:44.842803 9756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260812 00:38:44.843333 9898 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Bootstrap starting.
I20260812 00:38:44.872234 9898 log.cc:824] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Log is configured to *not* fsync() on all Append() calls
I20260812 00:38:44.881517 9756 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:33587
I20260812 00:38:44.881585 10134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:33587 every 8 connection(s)
I20260812 00:38:44.883006 9756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:38:44.886140 9898 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 00:38:44.886920 9898 tablet_bootstrap.cc:492] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Bootstrap complete.
I20260812 00:38:44.887019 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 9756
I20260812 00:38:44.887815 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:37911
--local_ip_for_outbound_sockets=127.7.244.66
--tserver_master_addrs=127.7.244.126:42765
--webserver_port=33873
--webserver_interface=127.7.244.66
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:38:44.888666 9898 ts_tablet_manager.cc:1397] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Time spent bootstrapping tablet: real 0.046s user 0.033s sys 0.008s
I20260812 00:38:44.888825 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:44.886067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:47565)
0812 00:38:44.887262 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:44.887579 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":479,"threads_started":1}
W20260812 00:38:44.889562 10138 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:42765 (0 consecutive failures): Network error: Failed to ping master at 127.7.244.126:42765: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
I20260812 00:38:44.890643 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:44.890191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:48559)
0812 00:38:44.890380 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:44.890485 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:38:44.891966 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:44.891481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:35723)
0812 00:38:44.891661 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:44.891777 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 00:38:44.892319 10138 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 00:38:44.896533 9898 raft_consensus.cc:348] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:44.897583 9898 raft_consensus.cc:729] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da292be6ba9445d180916e1368fe48bf, State: Initialized, Role: FOLLOWER
I20260812 00:38:44.898314 9898 consensus_queue.cc:260] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:44.900205 9898 ts_tablet_manager.cc:1428] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf: Time spent starting tablet: real 0.011s user 0.005s sys 0.008s
I20260812 00:38:44.900439 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:44.899931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:57969)
0812 00:38:44.900147 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:44.900263 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 00:38:44.901569 10144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:44.901139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:38151)
0812 00:38:44.901328 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:44.901416 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 00:38:45.251773 10145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:38:45.252106 10145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:38:45.252204 10145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:38:45.260035 10145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:38:45.260175 10145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:38:45.260313 10145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:38:45.270664 10145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:37911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=33873
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:38:45.272043 10145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:38:45.273638 10145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:38:45.288985 10167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:45.288975 10165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:45.288981 10162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:38:45.289811 10145 server_base.cc:1034] running on GCE node
I20260812 00:38:45.290525 10145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:38:45.291635 10145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:38:45.292901 10145 hybrid_clock.cc:648] HybridClock initialized: now 1786495125292818 us; error 65 us; skew 500 ppm
I20260812 00:38:45.293267 10145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:38:45 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495125, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:38:45.296813 10145 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:38:45.298586 10145 webserver.cc:460] Webserver started at http://127.7.244.66:33873/ using document root <none> and password file <none>
I20260812 00:38:45.299345 10145 fs_manager.cc:359] Metadata directory not provided
I20260812 00:38:45.299481 10145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:38:45.305692 10145 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 00:38:45.308830 10184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:45.310359 10145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:38:45.310557 10145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "39d5c5787a464663aab8ee580d96e0f2"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "21b08c193b95a4b2bd0269714d558298"
server_key_iv: "1a86e1c86d70f62b7d56cf297b069762"
server_key_version: "encryptionkey@0"
I20260812 00:38:45.311628 10145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:38:45.331365 10145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:38:45.336078 10145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:38:45.336371 10145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:38:45.337558 10145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:38:45.339074 10145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:38:45.339197 10145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:45.339313 10145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:38:45.339373 10145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:45.372387 10145 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:37911
I20260812 00:38:45.372475 10418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:37911 every 8 connection(s)
I20260812 00:38:45.374001 10145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:38:45.379208 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:45.377213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:60513)
0812 00:38:45.378035 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:45.378290 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":294,"threads_started":1}
W20260812 00:38:45.380069 10425 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:42765 (0 consecutive failures): Network error: Failed to ping master at 127.7.244.126:42765: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
I20260812 00:38:45.380403 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 10145
I20260812 00:38:45.381124 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:45.380667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:48061)
0812 00:38:45.380860 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:45.380971 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 00:38:45.381181 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:34625
--local_ip_for_outbound_sockets=127.7.244.67
--tserver_master_addrs=127.7.244.126:42765
--webserver_port=42033
--webserver_interface=127.7.244.67
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:38:45.382216 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:45.381777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:60285)
0812 00:38:45.381960 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:45.382062 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 00:38:45.382491 10425 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 00:38:45.742972 10429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:38:45.743309 10429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:38:45.743423 10429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:38:45.750917 10429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:38:45.751050 10429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:38:45.751204 10429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:38:45.761626 10429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:34625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=42033
--tserver_master_addrs=127.7.244.126:42765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:38:45.763082 10429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:38:45.764487 10429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:38:45.778738 10441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:45.778669 10442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:38:45.778702 10446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:38:45.780381 10429 server_base.cc:1034] running on GCE node
I20260812 00:38:45.781116 10429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:38:45.782311 10429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:38:45.783584 10429 hybrid_clock.cc:648] HybridClock initialized: now 1786495125783519 us; error 45 us; skew 500 ppm
I20260812 00:38:45.783958 10429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:38:45 dist-test-slave-gjw7 krb5kdc[7390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495125, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:38:45.787559 10429 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:38:45.789435 10429 webserver.cc:460] Webserver started at http://127.7.244.67:42033/ using document root <none> and password file <none>
I20260812 00:38:45.790261 10429 fs_manager.cc:359] Metadata directory not provided
I20260812 00:38:45.790396 10429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:38:45.796789 10429 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 00:38:45.800312 10464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:45.801941 10429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 00:38:45.802135 10429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "7bfba26eb500425a95e74f89672e3f33"
format_stamp: "Formatted at 2026-08-12 00:37:59 on dist-test-slave-gjw7"
server_key: "e0f3bc8b28139f04bcb00d9956db082b"
server_key_iv: "06b7400af251f8d900e6cda513a53e01"
server_key_version: "encryptionkey@0"
I20260812 00:38:45.803194 10429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:38:45.826570 10429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:38:45.831584 10429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:38:45.831905 10429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:38:45.833053 10429 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:38:45.834735 10429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:38:45.834856 10429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:45.834982 10429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:38:45.835055 10429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:38:45.868227 10429 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:34625
I20260812 00:38:45.868285 10717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:34625 every 8 connection(s)
I20260812 00:38:45.869879 10429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:38:45.873413 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 10429
I20260812 00:38:45.875895 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:45.873366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:34899)
0812 00:38:45.874331 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:45.874615 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":379,"threads_started":1}
W20260812 00:38:45.876740 10718 heartbeater.cc:644] Failed to heartbeat to 127.7.244.126:42765 (0 consecutive failures): Network error: Failed to ping master at 127.7.244.126:42765: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
I20260812 00:38:45.877748 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:45.877380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:60081)
0812 00:38:45.877543 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:45.877621 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 00:38:45.878723 10723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:45.878318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:46863)
0812 00:38:45.878473 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:45.878569 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 00:38:45.878995 10718 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 00:38:45.903682 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:45.902696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:47419)
0812 00:38:45.903293 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:45.903437 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":298,"threads_started":1}
I20260812 00:38:46.270733 10734 raft_consensus.cc:482] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:38:46.271064 10734 raft_consensus.cc:504] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:46.272753 10734 leader_election.cc:288] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625), 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911)
Aug 12 00:38:46 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
Aug 12 00:38:46 dist-test-slave-gjw7 krb5kdc[7390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:38:46.297961 10742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:46.274076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:42451 (local address 127.7.244.66:37911)
0812 00:38:46.275235 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:46.275255 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:46.275301 (+ 46us) server_negotiation.cc:407] Connection header received
0812 00:38:46.277152 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:46.277166 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:46.277685 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:46.278007 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:46.280229 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:46.281515 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:46.283537 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:46.284220 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:46.284428 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:46.289114 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:46.289128 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:46.289145 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:46.289198 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:46.293329 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:46.294478 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:46.294487 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:46.294494 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:46.294719 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:46.295282 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:46.295287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:46.295289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:46.295906 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:46.296107 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:46.297052 (+ 945us) server_negotiation.cc:299] Negotiation successful
0812 00:38:46.297493 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":931,"thread_start_us":799,"threads_started":1}
I20260812 00:38:46.297978 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:46.273735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:37911 (local address 127.7.244.65:42451)
0812 00:38:46.274898 (+ 1163us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:46.274918 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:46.275042 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:46.278255 (+ 3213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:46.278267 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:46.279152 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:38:46.279989 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:46.280014 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:46.281735 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:46.281740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:46.283354 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:46.283370 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:46.284843 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:46.284853 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:46.284999 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:46.285659 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:38:46.285678 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:38:46.288725 (+ 3047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:38:46.293590 (+ 4865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:46.293594 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:46.293606 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:46.294286 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:46.294896 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:46.294900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:46.294903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:46.295147 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:46.296104 (+ 957us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:38:46.296112 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:38:46.296483 (+ 371us) client_negotiation.cc:769] Sending connection context
0812 00:38:46.296768 (+ 285us) client_negotiation.cc:240] Negotiation successful
0812 00:38:46.297477 (+ 709us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":203,"spinlock_wait_cycles":59264,"thread_start_us":780,"threads_started":1}
I20260812 00:38:46.297979 10740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:46.273503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:55505 (local address 127.7.244.67:34625)
0812 00:38:46.275548 (+ 2045us) server_negotiation.cc:206] Beginning negotiation
0812 00:38:46.275560 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:38:46.275598 (+ 38us) server_negotiation.cc:407] Connection header received
0812 00:38:46.277401 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:38:46.277411 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:38:46.277777 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:38:46.278099 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:38:46.280225 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:46.281514 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:46.283529 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:38:46.284205 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:38:46.284419 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:46.288896 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:38:46.288914 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:38:46.288939 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:38:46.289017 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:38:46.293327 (+ 4310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:46.294487 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:46.294495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:46.294502 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:46.294725 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:38:46.295289 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:38:46.295293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:38:46.295295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:38:46.295900 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:38:46.296107 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 00:38:46.297054 (+ 947us) server_negotiation.cc:299] Negotiation successful
0812 00:38:46.297550 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1889,"thread_start_us":1737,"threads_started":1}
I20260812 00:38:46.298295 10732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:46.273136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:34625 (local address 127.7.244.65:55505)
0812 00:38:46.273385 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:46.273427 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:38:46.274022 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:38:46.278256 (+ 4234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:38:46.278268 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:38:46.279155 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:38:46.279990 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:46.280014 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:46.281733 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:46.281738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:46.283358 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:38:46.283373 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:38:46.284474 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:38:46.284482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:38:46.284736 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:38:46.285569 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:38:46.285600 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:38:46.288433 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:38:46.293571 (+ 5138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:46.293582 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:46.293598 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:46.294286 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:46.294913 (+ 627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:38:46.294918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:38:46.294920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:38:46.295158 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:38:46.296105 (+ 947us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:38:46.296113 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:38:46.296483 (+ 370us) client_negotiation.cc:769] Sending connection context
0812 00:38:46.296768 (+ 285us) client_negotiation.cc:240] Negotiation successful
0812 00:38:46.297825 (+ 1057us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":758,"spinlock_wait_cycles":40448}
W20260812 00:38:46.301699 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
W20260812 00:38:46.302064 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
I20260812 00:38:46.302178 9886 leader_election.cc:302] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da292be6ba9445d180916e1368fe48bf; no voters: 39d5c5787a464663aab8ee580d96e0f2, 7bfba26eb500425a95e74f89672e3f33
I20260812 00:38:46.302635 10734 raft_consensus.cc:2726] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:38:46.384124 10743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:46.383140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:50125)
0812 00:38:46.383671 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:46.383859 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":250,"threads_started":1}
I20260812 00:38:46.880460 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:46.879608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:41485)
0812 00:38:46.880078 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:46.880219 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":230,"threads_started":1}
I20260812 00:38:46.905859 10745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:46.904963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:38785)
0812 00:38:46.905553 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:46.905657 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":334,"threads_started":1}
I20260812 00:38:47.386125 10747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:47.385246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:39665)
0812 00:38:47.385834 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:47.385952 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":306,"threads_started":1}
I20260812 00:38:47.883052 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:47.881577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:58325)
0812 00:38:47.882406 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:47.882607 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":363,"threads_started":1}
I20260812 00:38:47.907820 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:47.906846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:46361)
0812 00:38:47.907459 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:47.907572 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":271,"threads_started":1}
I20260812 00:38:48.348304 10751 raft_consensus.cc:482] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:38:48.348533 10751 raft_consensus.cc:504] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:48.349253 10751 leader_election.cc:288] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625), 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911)
W20260812 00:38:48.350489 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
W20260812 00:38:48.350755 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
I20260812 00:38:48.350850 9886 leader_election.cc:302] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da292be6ba9445d180916e1368fe48bf; no voters: 39d5c5787a464663aab8ee580d96e0f2, 7bfba26eb500425a95e74f89672e3f33
I20260812 00:38:48.351156 10751 raft_consensus.cc:2726] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:38:48.387866 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:48.387105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:34147)
0812 00:38:48.387607 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:48.387689 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":294,"threads_started":1}
I20260812 00:38:48.885115 10753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:48.884355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:46833)
0812 00:38:48.884855 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:48.884957 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":243,"threads_started":1}
I20260812 00:38:48.909857 10756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:48.909002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:35317)
0812 00:38:48.909573 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:48.909676 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":291,"threads_started":1}
I20260812 00:38:49.389840 10757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:49.388980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:37583)
0812 00:38:49.389543 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:49.389643 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":290,"threads_started":1}
I20260812 00:38:49.887004 10760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:49.886240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:37693)
0812 00:38:49.886714 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:49.886823 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":230,"threads_started":1}
I20260812 00:38:49.911954 10761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:49.911064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:58161)
0812 00:38:49.911666 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:49.911762 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":353,"threads_started":1}
I20260812 00:38:50.391767 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:50.390953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:53441)
0812 00:38:50.391476 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:50.391586 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":274,"threads_started":1}
I20260812 00:38:50.772609 10765 raft_consensus.cc:482] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:38:50.772851 10765 raft_consensus.cc:504] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:50.773554 10765 leader_election.cc:288] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625), 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911)
W20260812 00:38:50.774643 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
W20260812 00:38:50.774853 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
I20260812 00:38:50.774953 9886 leader_election.cc:302] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da292be6ba9445d180916e1368fe48bf; no voters: 39d5c5787a464663aab8ee580d96e0f2, 7bfba26eb500425a95e74f89672e3f33
I20260812 00:38:50.775198 10765 raft_consensus.cc:2726] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:38:50.889026 10766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:50.888138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:36105)
0812 00:38:50.888726 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:50.888842 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":334,"threads_started":1}
I20260812 00:38:50.913919 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:50.912846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:49909)
0812 00:38:50.913473 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:50.913657 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":279,"threads_started":1}
I20260812 00:38:51.393586 10774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:51.392749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:39131)
0812 00:38:51.393232 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:51.393356 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":239,"threads_started":1}
I20260812 00:38:51.891002 10775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:51.890115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:58197)
0812 00:38:51.890718 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:51.890808 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":351,"threads_started":1}
I20260812 00:38:51.916353 10778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:51.915046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:37323)
0812 00:38:51.916036 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:51.916145 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":730,"threads_started":1}
I20260812 00:38:52.395704 10786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:52.394685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:41467)
0812 00:38:52.395375 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:52.395493 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":300,"threads_started":1}
I20260812 00:38:52.892940 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:52.892016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:43433)
0812 00:38:52.892619 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:52.892711 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":255,"threads_started":1}
I20260812 00:38:52.918342 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:52.917418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:33423)
0812 00:38:52.918006 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:52.918116 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":305,"threads_started":1}
I20260812 00:38:53.018507 10792 raft_consensus.cc:482] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:38:53.018739 10792 raft_consensus.cc:504] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:53.019424 10792 leader_election.cc:288] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625), 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911)
W20260812 00:38:53.020434 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
W20260812 00:38:53.020704 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
I20260812 00:38:53.020802 9886 leader_election.cc:302] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da292be6ba9445d180916e1368fe48bf; no voters: 39d5c5787a464663aab8ee580d96e0f2, 7bfba26eb500425a95e74f89672e3f33
I20260812 00:38:53.021085 10792 raft_consensus.cc:2726] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:38:53.397559 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:53.396738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:42113)
0812 00:38:53.397215 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:53.397342 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":236,"threads_started":1}
I20260812 00:38:53.894958 10804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:53.894139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:41581)
0812 00:38:53.894637 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:53.894744 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":275,"threads_started":1}
I20260812 00:38:53.920053 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:53.919281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:43335)
0812 00:38:53.919759 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:53.919847 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":237,"threads_started":1}
I20260812 00:38:54.399421 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:54.398597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:56637)
0812 00:38:54.399121 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:54.399220 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":255,"threads_started":1}
I20260812 00:38:54.896929 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:54.896094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:40481)
0812 00:38:54.896628 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:54.896747 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":272,"threads_started":1}
I20260812 00:38:54.922231 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:54.921354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:41497)
0812 00:38:54.921882 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:54.921997 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":271,"threads_started":1}
I20260812 00:38:55.401361 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:55.400488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:57955)
0812 00:38:55.401034 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:55.401129 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":326,"threads_started":1}
I20260812 00:38:55.898823 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:55.898040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:51713)
0812 00:38:55.898542 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:55.898638 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":254,"threads_started":1}
I20260812 00:38:55.924314 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:55.923384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:57371)
0812 00:38:55.923957 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:55.924072 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":296,"threads_started":1}
I20260812 00:38:56.403417 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:56.402549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:36069)
0812 00:38:56.403125 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:56.403229 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":282,"threads_started":1}
I20260812 00:38:56.901821 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:56.900451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:55291)
0812 00:38:56.901426 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:56.901575 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":468,"threads_started":1}
I20260812 00:38:56.927253 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:56.925869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:51331)
0812 00:38:56.926730 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:56.926921 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":412,"threads_started":1}
I20260812 00:38:57.406663 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:57.405028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:46967)
0812 00:38:57.406012 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:57.406227 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":500,"threads_started":1}
I20260812 00:38:57.903801 10833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:57.903034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:34537)
0812 00:38:57.903509 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:57.903600 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":234,"threads_started":1}
I20260812 00:38:57.929401 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:57.928538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:35457)
0812 00:38:57.929031 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:57.929137 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":238,"threads_started":1}
I20260812 00:38:58.409103 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:58.408257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:60479)
0812 00:38:58.408739 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:58.408855 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":237,"threads_started":1}
I20260812 00:38:58.905812 10841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:58.904887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:36943)
0812 00:38:58.905493 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:58.905607 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":297,"threads_started":1}
I20260812 00:38:58.931239 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:58.930399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:56443)
0812 00:38:58.930927 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:58.931042 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":263,"threads_started":1}
I20260812 00:38:59.410985 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:59.410121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:40231)
0812 00:38:59.410669 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:59.410777 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":309,"threads_started":1}
I20260812 00:38:59.850353 10844 raft_consensus.cc:482] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:38:59.850562 10844 raft_consensus.cc:504] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da292be6ba9445d180916e1368fe48bf" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 33587 } } peers { permanent_uuid: "7bfba26eb500425a95e74f89672e3f33" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 34625 } } peers { permanent_uuid: "39d5c5787a464663aab8ee580d96e0f2" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 37911 } }
I20260812 00:38:59.851202 10844 leader_election.cc:288] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625), 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911)
W20260812 00:38:59.852355 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7bfba26eb500425a95e74f89672e3f33 (127.7.244.67:34625): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
W20260812 00:38:59.852577 9886 leader_election.cc:341] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 39d5c5787a464663aab8ee580d96e0f2 (127.7.244.66:37911): Not found: Tablet not found: 972ec7063b2b49d686a6350fdb5c8391
I20260812 00:38:59.852689 9886 leader_election.cc:302] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da292be6ba9445d180916e1368fe48bf; no voters: 39d5c5787a464663aab8ee580d96e0f2, 7bfba26eb500425a95e74f89672e3f33
I20260812 00:38:59.852981 10844 raft_consensus.cc:2726] T 972ec7063b2b49d686a6350fdb5c8391 P da292be6ba9445d180916e1368fe48bf [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 00:38:59.907816 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:59.906931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:40247)
0812 00:38:59.907508 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:59.907607 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":270,"threads_started":1}
I20260812 00:38:59.933805 10850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:38:59.932578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:55079)
0812 00:38:59.933345 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0812 00:38:59.933532 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":323,"threads_started":1}
I20260812 00:39:00.413230 10852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:00.412289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.66:59097)
0812 00:39:00.412855 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:00.413016 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":275,"threads_started":1}
I20260812 00:39:00.882047 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 9756
I20260812 00:39:00.901222 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:00.883557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39418 (local address 127.7.244.65:33587)
0812 00:39:00.884913 (+ 1356us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:00.884923 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:00.884962 (+ 39us) server_negotiation.cc:407] Connection header received
0812 00:39:00.885033 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:00.885040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:00.885276 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:00.885475 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:00.887598 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:00.888699 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:00.890224 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:00.890895 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:00.891013 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:00.894238 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:00.894263 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:00.894283 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:00.894331 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:00.897640 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:00.898654 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:00.898664 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:00.898666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:00.898878 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:00.899396 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:00.899401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:00.899403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:00.899847 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:00.900121 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:00.900642 (+ 521us) server_negotiation.cc:299] Negotiation successful
0812 00:39:00.900936 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":916,"threads_started":1}
I20260812 00:39:00.902778 10126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:00.910000 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:00.909148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.67:44677)
0812 00:39:00.909717 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:00.909820 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":313,"threads_started":1}
I20260812 00:39:00.994402 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:00.993274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:42765 (local address 127.7.244.65:34495)
0812 00:39:00.993832 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:00.994047 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.126:42765: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":250,"threads_started":1}
I20260812 00:39:00.994464 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 9756
I20260812 00:39:01.023691 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 10145
I20260812 00:39:01.038632 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:01.024508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.7.244.66:37911)
0812 00:39:01.025041 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:01.025045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:01.025061 (+ 16us) server_negotiation.cc:407] Connection header received
0812 00:39:01.025121 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:01.025124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:01.025187 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:01.025319 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:01.026886 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:01.027686 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:01.029126 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:01.029640 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:01.029767 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:01.032624 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:01.032629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:01.032633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:01.032677 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:01.035495 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:01.036387 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:01.036393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:01.036396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:01.036572 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:01.036992 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:01.036996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:01.036998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:01.037353 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:01.037665 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:01.037997 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 00:39:01.038241 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":300,"threads_started":1}
I20260812 00:39:01.039430 10411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:01.124754 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 10145
I20260812 00:39:01.154541 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 10429
I20260812 00:39:01.169458 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:01.155508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37856 (local address 127.7.244.67:34625)
0812 00:39:01.156150 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:01.156155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:01.156172 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:39:01.156231 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:01.156235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:01.156297 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:01.156412 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:01.158026 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:01.158883 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:01.160218 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:01.160686 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:01.160811 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:01.163550 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:01.163556 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:01.163559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:01.163600 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:01.166439 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:01.167314 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:01.167319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:01.167321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:01.167473 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:01.167893 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:01.167897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:01.167899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:01.168207 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:01.168523 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:01.168789 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 00:39:01.169035 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":349,"threads_started":1}
I20260812 00:39:01.170311 10710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:01.255079 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 10429
2026-08-12T00:39:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65388 ms)
[----------] 30 tests from SecurityITest (316392 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:01 dist-test-slave-gjw7 krb5kdc[10872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:01 dist-test-slave-gjw7 krb5kdc[10872](info): set up 2 sockets
Aug 12 00:39:01 dist-test-slave-gjw7 krb5kdc[10872](info): commencing operation
krb5kdc: starting...
W20260812 00:39:02.614540 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.303s user 0.007s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:02 dist-test-slave-gjw7 krb5kdc[10872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:39:02.933684 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44111
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:35635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:44111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:03.304507 10892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:03.304854 10892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:03.304999 10892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:03.312631 10892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:03.312767 10892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:03.312829 10892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:03.312922 10892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:03.312980 10892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:03.323446 10892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:44111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:03.324908 10892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:03.326370 10892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:03.341102 10902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:03.341053 10905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:03.341132 10903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:03.341337 10892 server_base.cc:1034] running on GCE node
I20260812 00:39:03.342434 10892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:03.343957 10892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:03.345173 10892 hybrid_clock.cc:648] HybridClock initialized: now 1786495143345121 us; error 39 us; skew 500 ppm
I20260812 00:39:03.345557 10892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:03 dist-test-slave-gjw7 krb5kdc[10872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:03.351719 10892 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:39:03.353655 10892 webserver.cc:460] Webserver started at http://127.7.244.126:35749/ using document root <none> and password file <none>
I20260812 00:39:03.354427 10892 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:03.354553 10892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:03.354866 10892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:03.358340 10892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "45bce2caab8745f0a83fe12beda9c146"
format_stamp: "Formatted at 2026-08-12 00:39:03 on dist-test-slave-gjw7"
server_key: "730edd6aa45ed45fefaa1f7a4a66955b"
server_key_iv: "a653ab9b7439e085af771e76124edea5"
server_key_version: "encryptionkey@0"
I20260812 00:39:03.359194 10892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "45bce2caab8745f0a83fe12beda9c146"
format_stamp: "Formatted at 2026-08-12 00:39:03 on dist-test-slave-gjw7"
server_key: "730edd6aa45ed45fefaa1f7a4a66955b"
server_key_iv: "a653ab9b7439e085af771e76124edea5"
server_key_version: "encryptionkey@0"
I20260812 00:39:03.365152 10892 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:39:03.369149 10916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:03.370828 10892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:39:03.371030 10892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "45bce2caab8745f0a83fe12beda9c146"
format_stamp: "Formatted at 2026-08-12 00:39:03 on dist-test-slave-gjw7"
server_key: "730edd6aa45ed45fefaa1f7a4a66955b"
server_key_iv: "a653ab9b7439e085af771e76124edea5"
server_key_version: "encryptionkey@0"
I20260812 00:39:03.371245 10892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:03.392657 10892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:03.397938 10892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:03.398268 10892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:03.419384 10892 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:44111
I20260812 00:39:03.419404 11023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:44111 every 8 connection(s)
I20260812 00:39:03.421037 10892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:03.425382 11026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:03.425446 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 10892
I20260812 00:39:03.425649 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:03.426554 8145 external_mini_cluster.cc:1428] Setting key 5924f7408e74fe75c5803550604cbf71
I20260812 00:39:03.441205 11026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146: Bootstrap starting.
Aug 12 00:39:03 dist-test-slave-gjw7 krb5kdc[10872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:03.445080 11026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:03.446331 11026 log.cc:824] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:03.450161 11026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146: No bootstrap required, opened a new log
I20260812 00:39:03.452764 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:03.428787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50544 (local address 127.7.244.126:44111)
0812 00:39:03.429654 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:03.429666 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:03.429716 (+ 50us) server_negotiation.cc:407] Connection header received
0812 00:39:03.431383 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:03.431414 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:03.431826 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:03.432178 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:03.434148 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:03.435539 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:03.437650 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:03.438511 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:03.438682 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:03.444076 (+ 5394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:03.444087 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:03.444104 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:03.444153 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:03.447908 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:03.448920 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:03.448929 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:03.448935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:03.449134 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:03.449782 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:03.449787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:03.449789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:03.450439 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:03.450692 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:03.451208 (+ 516us) server_negotiation.cc:299] Negotiation successful
0812 00:39:03.451595 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":651,"thread_start_us":403,"threads_started":1}
I20260812 00:39:03.458884 11026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bce2caab8745f0a83fe12beda9c146" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44111 } }
I20260812 00:39:03.459339 11026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:03.459447 11026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45bce2caab8745f0a83fe12beda9c146, State: Initialized, Role: FOLLOWER
I20260812 00:39:03.460062 11026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bce2caab8745f0a83fe12beda9c146" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44111 } }
I20260812 00:39:03.460304 11026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:03.460438 11026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:03.460608 11026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:03.463346 11026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bce2caab8745f0a83fe12beda9c146" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44111 } }
I20260812 00:39:03.463927 11026 leader_election.cc:302] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45bce2caab8745f0a83fe12beda9c146; no voters:
I20260812 00:39:03.464555 11026 leader_election.cc:288] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:03.464696 11046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:03.465404 11046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [term 1 LEADER]: Becoming Leader. State: Replica: 45bce2caab8745f0a83fe12beda9c146, State: Running, Role: LEADER
I20260812 00:39:03.465960 11046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bce2caab8745f0a83fe12beda9c146" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44111 } }
I20260812 00:39:03.466794 11026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:03.468927 11048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45bce2caab8745f0a83fe12beda9c146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bce2caab8745f0a83fe12beda9c146" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44111 } } }
I20260812 00:39:03.469663 11049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45bce2caab8745f0a83fe12beda9c146. Latest consensus state: current_term: 1 leader_uuid: "45bce2caab8745f0a83fe12beda9c146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45bce2caab8745f0a83fe12beda9c146" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44111 } } }
I20260812 00:39:03.470001 11049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:03.470009 11048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:03.475317 11054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:03.480096 11054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:03.490275 11054 catalog_manager.cc:1347] Generated new cluster ID: b3509728fb06466bb793d37712ebc1a5
I20260812 00:39:03.490509 11054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:03.533731 11054 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:03.535152 11054 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:03.545857 11054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146: Generated new TSK 0
I20260812 00:39:03.546872 11054 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:39:03.621059 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44111
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:35635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:03.993178 11087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:03.993577 11087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:03.993721 11087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:04.001478 11087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:04.001610 11087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:04.001765 11087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:39:04.012348 11087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:04.013909 11087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:04.015372 11087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:39:04.030820 11087 server_base.cc:1034] running on GCE node
W20260812 00:39:04.033399 11095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:04.033399 11099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:04.033399 11097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:04.034588 11087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:04.035713 11087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:04.036967 11087 hybrid_clock.cc:648] HybridClock initialized: now 1786495144036915 us; error 64 us; skew 500 ppm
I20260812 00:39:04.037391 11087 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:04 dist-test-slave-gjw7 krb5kdc[10872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:04.040932 11087 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:39:04.042735 11087 webserver.cc:460] Webserver started at http://127.7.244.65:40897/ using document root <none> and password file <none>
I20260812 00:39:04.043509 11087 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:04.043633 11087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:04.043968 11087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:04.047410 11087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "d086567127114dcabaec6b9feabeda07"
format_stamp: "Formatted at 2026-08-12 00:39:04 on dist-test-slave-gjw7"
server_key: "76314bf77015e4992cc0b409700def76"
server_key_iv: "907e0e3cbf256641736c177841538f94"
server_key_version: "encryptionkey@0"
I20260812 00:39:04.048316 11087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "d086567127114dcabaec6b9feabeda07"
format_stamp: "Formatted at 2026-08-12 00:39:04 on dist-test-slave-gjw7"
server_key: "76314bf77015e4992cc0b409700def76"
server_key_iv: "907e0e3cbf256641736c177841538f94"
server_key_version: "encryptionkey@0"
I20260812 00:39:04.054302 11087 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:39:04.058405 11116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:04.060004 11087 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:39:04.060187 11087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "d086567127114dcabaec6b9feabeda07"
format_stamp: "Formatted at 2026-08-12 00:39:04 on dist-test-slave-gjw7"
server_key: "76314bf77015e4992cc0b409700def76"
server_key_iv: "907e0e3cbf256641736c177841538f94"
server_key_version: "encryptionkey@0"
I20260812 00:39:04.060397 11087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:04.073258 11087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:04.078085 11087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:04.078361 11087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:04.079389 11087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:04.080832 11087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:04.080909 11087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:04.081004 11087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:04.081046 11087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:04.113938 11087 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:34495
I20260812 00:39:04.113993 11324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:34495 every 8 connection(s)
I20260812 00:39:04.115252 11087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:39:04.123231 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 11087
I20260812 00:39:04.123448 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:39:04.124127 8145 external_mini_cluster.cc:1428] Setting key 5c1b61dd5a3fceb306ea9e235a27c55c
Aug 12 00:39:04 dist-test-slave-gjw7 krb5kdc[10872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:04.138407 11329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:04.117918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:60887 (local address 127.7.244.126:44111)
0812 00:39:04.118385 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:04.118390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:04.119410 (+ 1020us) server_negotiation.cc:407] Connection header received
0812 00:39:04.121399 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:04.121406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:04.121510 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:04.121611 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:04.123274 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:04.124111 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:04.125963 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:04.126498 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:04.126640 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:04.130615 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:04.130623 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:04.130628 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:04.130674 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:04.134255 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:04.135301 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:04.135308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:04.135311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:04.135484 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:04.136153 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:04.136158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:04.136161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:04.136539 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:04.136695 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:04.137780 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0812 00:39:04.138052 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":234,"threads_started":1}
I20260812 00:39:04.139699 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:04.118490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44111 (local address 127.7.244.65:60887)
0812 00:39:04.119197 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:04.119277 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:04.121061 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:04.121890 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:04.121899 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:04.122472 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:04.123085 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:04.123105 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:04.124290 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:04.124294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:04.125797 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:04.125813 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:04.126757 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:04.126763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:04.126995 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:04.127709 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:04.127735 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:04.130289 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:04.134505 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:04.134515 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:04.134533 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:04.135139 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:04.135708 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:04.135713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:04.135715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:04.136012 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:04.136729 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:04.136738 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:04.137225 (+ 487us) client_negotiation.cc:769] Sending connection context
0812 00:39:04.137631 (+ 406us) client_negotiation.cc:240] Negotiation successful
0812 00:39:04.138153 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":271,"threads_started":1}
I20260812 00:39:04.141651 11326 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44111
I20260812 00:39:04.142098 11326 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:04.143224 11326 heartbeater.cc:511] Master 127.7.244.126:44111 requested a full tablet report, sending...
I20260812 00:39:04.145947 10959 ts_manager.cc:194] Registered new tserver with Master: d086567127114dcabaec6b9feabeda07 (127.7.244.65:34495)
I20260812 00:39:04.147521 10959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:60887
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:39:04.197706 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44111
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:35635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:04.564606 11337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:04.564985 11337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:04.565130 11337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:04.573618 11337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:04.573796 11337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:04.573962 11337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:39:04.584657 11337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:04.586169 11337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:04.587699 11337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:04.601907 11346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:04.601943 11349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:04.601967 11347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:04.602767 11337 server_base.cc:1034] running on GCE node
I20260812 00:39:04.603499 11337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:04.604573 11337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:04.605854 11337 hybrid_clock.cc:648] HybridClock initialized: now 1786495144605766 us; error 45 us; skew 500 ppm
I20260812 00:39:04.606236 11337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:04 dist-test-slave-gjw7 krb5kdc[10872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:04.609567 11337 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:39:04.611275 11337 webserver.cc:460] Webserver started at http://127.7.244.66:41589/ using document root <none> and password file <none>
I20260812 00:39:04.612046 11337 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:04.612185 11337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:04.612524 11337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:04.615741 11337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "8e4c21b44b314f818649a08592ca3521"
format_stamp: "Formatted at 2026-08-12 00:39:04 on dist-test-slave-gjw7"
server_key: "57802c56e1bf0b5a0317dc127faddd68"
server_key_iv: "72b3aca480cac0a86c5ce2c2c1f00bd9"
server_key_version: "encryptionkey@0"
I20260812 00:39:04.616643 11337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "8e4c21b44b314f818649a08592ca3521"
format_stamp: "Formatted at 2026-08-12 00:39:04 on dist-test-slave-gjw7"
server_key: "57802c56e1bf0b5a0317dc127faddd68"
server_key_iv: "72b3aca480cac0a86c5ce2c2c1f00bd9"
server_key_version: "encryptionkey@0"
I20260812 00:39:04.622318 11337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260812 00:39:04.626446 11368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:04.628199 11337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:39:04.628412 11337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "8e4c21b44b314f818649a08592ca3521"
format_stamp: "Formatted at 2026-08-12 00:39:04 on dist-test-slave-gjw7"
server_key: "57802c56e1bf0b5a0317dc127faddd68"
server_key_iv: "72b3aca480cac0a86c5ce2c2c1f00bd9"
server_key_version: "encryptionkey@0"
I20260812 00:39:04.628657 11337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:04.653532 11337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:04.659291 11337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:04.659605 11337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:04.660703 11337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:04.662241 11337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:04.662333 11337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:04.662463 11337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:04.662518 11337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:04.695794 11337 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:43457
I20260812 00:39:04.695859 11584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:43457 every 8 connection(s)
I20260812 00:39:04.697228 11337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:39:04.699513 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 11337
I20260812 00:39:04.699723 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:39:04.700317 8145 external_mini_cluster.cc:1428] Setting key 7daa067ccb952170293df6385587f742
Aug 12 00:39:04 dist-test-slave-gjw7 krb5kdc[10872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:04.720950 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:04.700158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:53421 (local address 127.7.244.126:44111)
0812 00:39:04.700655 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:04.700660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:04.702161 (+ 1501us) server_negotiation.cc:407] Connection header received
0812 00:39:04.704361 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:04.704369 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:04.704464 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:04.704589 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:04.706518 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:04.707554 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:04.709348 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:04.709976 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:04.710116 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:04.714321 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:04.714328 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:04.714331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:04.714372 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:04.717475 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:04.718346 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:04.718352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:04.718354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:04.718506 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:04.718996 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:04.719000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:04.719002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:04.719308 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:04.719463 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:04.720269 (+ 806us) server_negotiation.cc:299] Negotiation successful
0812 00:39:04.720552 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":250,"threads_started":1}
I20260812 00:39:04.722084 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:04.700966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44111 (local address 127.7.244.66:53421)
0812 00:39:04.701968 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:04.702027 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:04.704005 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:04.704905 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:04.704915 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:04.705599 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:04.706286 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:04.706310 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:04.707777 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:04.707782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:04.709126 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:04.709143 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:04.710234 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:04.710240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:04.710537 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:04.711328 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:04.711358 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:04.714025 (+ 2667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:04.717657 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:04.717668 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:04.717687 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:04.718193 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:04.718629 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:04.718634 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:04.718636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:04.718853 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:04.719449 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:04.719458 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:04.719856 (+ 398us) client_negotiation.cc:769] Sending connection context
0812 00:39:04.720131 (+ 275us) client_negotiation.cc:240] Negotiation successful
0812 00:39:04.720735 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":398,"threads_started":1}
I20260812 00:39:04.723904 11586 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44111
I20260812 00:39:04.724242 11586 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:04.725036 11586 heartbeater.cc:511] Master 127.7.244.126:44111 requested a full tablet report, sending...
I20260812 00:39:04.726995 10959 ts_manager.cc:194] Registered new tserver with Master: 8e4c21b44b314f818649a08592ca3521 (127.7.244.66:43457)
I20260812 00:39:04.728171 10959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:53421
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:39:04.769018 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44111
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:35635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:05.138994 11593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:05.139319 11593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:05.139456 11593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:05.147210 11593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:05.147356 11593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:05.147511 11593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:39:05.151924 11326 heartbeater.cc:503] Master 127.7.244.126:44111 was elected leader, sending a full tablet report...
I20260812 00:39:05.158419 11593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:05.159930 11593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:05.161446 11593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:05.175969 11612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:05.176273 11613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:05.176277 11619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:05.177109 11593 server_base.cc:1034] running on GCE node
I20260812 00:39:05.177832 11593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:05.178908 11593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:05.180114 11593 hybrid_clock.cc:648] HybridClock initialized: now 1786495145180047 us; error 60 us; skew 500 ppm
I20260812 00:39:05.180478 11593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:05 dist-test-slave-gjw7 krb5kdc[10872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:05.183851 11593 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:39:05.185679 11593 webserver.cc:460] Webserver started at http://127.7.244.67:46061/ using document root <none> and password file <none>
I20260812 00:39:05.186462 11593 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:05.186587 11593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:05.186898 11593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:05.190253 11593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "4d8b27d6880d4ace9219e41348bc0e62"
format_stamp: "Formatted at 2026-08-12 00:39:05 on dist-test-slave-gjw7"
server_key: "8b8c72be70dc967ccec040e6592b8f0d"
server_key_iv: "8c396e193961dcd5f200dfda542199ea"
server_key_version: "encryptionkey@0"
I20260812 00:39:05.191156 11593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "4d8b27d6880d4ace9219e41348bc0e62"
format_stamp: "Formatted at 2026-08-12 00:39:05 on dist-test-slave-gjw7"
server_key: "8b8c72be70dc967ccec040e6592b8f0d"
server_key_iv: "8c396e193961dcd5f200dfda542199ea"
server_key_version: "encryptionkey@0"
I20260812 00:39:05.197083 11593 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 00:39:05.201386 11631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:05.202984 11593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:39:05.203179 11593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "4d8b27d6880d4ace9219e41348bc0e62"
format_stamp: "Formatted at 2026-08-12 00:39:05 on dist-test-slave-gjw7"
server_key: "8b8c72be70dc967ccec040e6592b8f0d"
server_key_iv: "8c396e193961dcd5f200dfda542199ea"
server_key_version: "encryptionkey@0"
I20260812 00:39:05.203483 11593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:05.236194 11593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:05.241047 11593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:05.241550 11593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:05.242726 11593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:05.244300 11593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:05.244385 11593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:05.244506 11593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:05.244567 11593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:05.279112 11593 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:40577
I20260812 00:39:05.279157 11844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:40577 every 8 connection(s)
I20260812 00:39:05.280517 11593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:39:05.281615 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 11593
I20260812 00:39:05.281836 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:39:05.282413 8145 external_mini_cluster.cc:1428] Setting key a1a658945af6bc56e4ea6acc7301a527
Aug 12 00:39:05 dist-test-slave-gjw7 krb5kdc[10872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:05.303431 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.283704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:50239 (local address 127.7.244.126:44111)
0812 00:39:05.284403 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.284407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.285550 (+ 1143us) server_negotiation.cc:407] Connection header received
0812 00:39:05.287597 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.287603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.287697 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.287797 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:05.289717 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.290539 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.292279 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.292709 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.292840 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.296805 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:05.296812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:05.296815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:05.296857 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:05.300060 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:05.300961 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:05.300966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:05.300968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:05.301118 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:05.301721 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:05.301726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:05.301728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:05.302027 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:05.302152 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.302930 (+ 778us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.303131 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":457,"threads_started":1}
I20260812 00:39:05.304447 11848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.284510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44111 (local address 127.7.244.67:50239)
0812 00:39:05.285391 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.285439 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:05.287282 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:05.288111 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:05.288121 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:05.288784 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:05.289502 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.289526 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.290733 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.290737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.292086 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.292102 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.292996 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.293004 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.293260 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.294026 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:05.294053 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:05.296511 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:05.300249 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:05.300259 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:05.300275 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:05.300802 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:05.301325 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:05.301331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:05.301335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:05.301595 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:05.302131 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:05.302138 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:05.302540 (+ 402us) client_negotiation.cc:769] Sending connection context
0812 00:39:05.302801 (+ 261us) client_negotiation.cc:240] Negotiation successful
0812 00:39:05.303260 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":325,"threads_started":1}
I20260812 00:39:05.306064 11845 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44111
I20260812 00:39:05.306443 11845 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:05.307305 11845 heartbeater.cc:511] Master 127.7.244.126:44111 requested a full tablet report, sending...
I20260812 00:39:05.309113 10959 ts_manager.cc:194] Registered new tserver with Master: 4d8b27d6880d4ace9219e41348bc0e62 (127.7.244.67:40577)
I20260812 00:39:05.310343 10959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:50239
I20260812 00:39:05.312455 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:39:05.333405 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.317638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.7.244.126:44111)
0812 00:39:05.317829 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.317834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.318415 (+ 581us) server_negotiation.cc:407] Connection header received
0812 00:39:05.318601 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.318606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.318682 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.318779 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:05.320411 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.321176 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.322859 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.323327 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.323442 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.326468 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:05.326476 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:05.326480 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:05.326532 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:05.329953 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:05.330927 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:05.330932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:05.330935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:05.331091 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:05.331717 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:05.331722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:05.331724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:05.332157 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:05.332356 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.332807 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.333072 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 00:39:05.338573 10959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:39:05.345767 10959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:39:05.371877 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.359714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.7.244.65:34495)
0812 00:39:05.362012 (+ 2298us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.362023 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.362058 (+ 35us) server_negotiation.cc:407] Connection header received
0812 00:39:05.362175 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.362185 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.362478 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.362702 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:05.363587 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.365460 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.368895 (+ 3435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.370568 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.370699 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.370988 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.371365 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.371540 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2085,"thread_start_us":1926,"threads_started":1}
I20260812 00:39:05.371954 11868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.359380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34714)
0812 00:39:05.360385 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.360427 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:05.360615 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:05.362772 (+ 2157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:05.362782 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:05.362836 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:05.363348 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.363365 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.365619 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.365625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.368656 (+ 3031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.368674 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.370823 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.370830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.371070 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.371127 (+ 57us) client_negotiation.cc:769] Sending connection context
0812 00:39:05.371260 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 00:39:05.371485 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"mutex_wait_us":38,"thread_start_us":581,"threads_started":1}
I20260812 00:39:05.373528 11875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.361090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:43457 (local address 127.0.0.1:47194)
0812 00:39:05.363884 (+ 2794us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.363904 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:05.364042 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:05.365104 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:05.365107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:05.365129 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:05.365482 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.365495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.367598 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.367603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.370118 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.370140 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.372599 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.372606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.372806 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.372845 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 00:39:05.372944 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 00:39:05.373092 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2683,"mutex_wait_us":25,"thread_start_us":2531,"threads_started":1}
I20260812 00:39:05.374457 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.361661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.7.244.66:43457)
0812 00:39:05.363434 (+ 1773us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.363443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.364172 (+ 729us) server_negotiation.cc:407] Connection header received
0812 00:39:05.364282 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.364290 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.364543 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.364811 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:05.365693 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.367417 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.370331 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.371866 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.372210 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.372441 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.373932 (+ 1491us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.374133 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1538,"thread_start_us":1400,"threads_started":1}
I20260812 00:39:05.374503 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.359696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:40577 (local address 127.0.0.1:38932)
0812 00:39:05.361215 (+ 1519us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.361234 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:05.361384 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:05.366640 (+ 5256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:05.366644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:05.366672 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:05.367091 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.367101 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.369023 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.369029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.371261 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.371278 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.373696 (+ 2418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.373703 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.373871 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.373907 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 00:39:05.373973 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 00:39:05.374110 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1390,"mutex_wait_us":28,"thread_start_us":1227,"threads_started":1}
I20260812 00:39:05.375468 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.360428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38932 (local address 127.7.244.67:40577)
0812 00:39:05.366120 (+ 5692us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.366126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.366154 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:39:05.366236 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.366241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.366425 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.366617 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:05.367296 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.368824 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.371485 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.373452 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.374642 (+ 1190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.374900 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.374983 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.375118 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5534,"thread_start_us":5346,"threads_started":1}
I20260812 00:39:05.376984 11202 tablet_service.cc:1467] Processing CreateTablet for tablet 759409b9c2654ea1b02eaaf5c32265a0 (DEFAULT_TABLE table=test-table [id=5488ccd758804639870304530fba5cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:05.378854 11202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759409b9c2654ea1b02eaaf5c32265a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:05.379143 11468 tablet_service.cc:1467] Processing CreateTablet for tablet 759409b9c2654ea1b02eaaf5c32265a0 (DEFAULT_TABLE table=test-table [id=5488ccd758804639870304530fba5cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:05.379264 11724 tablet_service.cc:1467] Processing CreateTablet for tablet 759409b9c2654ea1b02eaaf5c32265a0 (DEFAULT_TABLE table=test-table [id=5488ccd758804639870304530fba5cc7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:05.380365 11468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759409b9c2654ea1b02eaaf5c32265a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:05.380738 11724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 759409b9c2654ea1b02eaaf5c32265a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:05.393922 11877 tablet_bootstrap.cc:492] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Bootstrap starting.
I20260812 00:39:05.394936 11878 tablet_bootstrap.cc:492] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Bootstrap starting.
I20260812 00:39:05.396816 11879 tablet_bootstrap.cc:492] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Bootstrap starting.
I20260812 00:39:05.398509 11877 tablet_bootstrap.cc:654] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:05.399223 11878 tablet_bootstrap.cc:654] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:05.399940 11877 log.cc:824] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:05.400272 11879 tablet_bootstrap.cc:654] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:05.400537 11878 log.cc:824] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:05.401615 11879 log.cc:824] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:05.404539 11878 tablet_bootstrap.cc:492] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: No bootstrap required, opened a new log
I20260812 00:39:05.404538 11877 tablet_bootstrap.cc:492] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: No bootstrap required, opened a new log
I20260812 00:39:05.404879 11877 ts_tablet_manager.cc:1397] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260812 00:39:05.404881 11878 ts_tablet_manager.cc:1397] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260812 00:39:05.406491 11879 tablet_bootstrap.cc:492] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: No bootstrap required, opened a new log
I20260812 00:39:05.406798 11879 ts_tablet_manager.cc:1397] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260812 00:39:05.415532 11879 raft_consensus.cc:348] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
I20260812 00:39:05.416028 11879 raft_consensus.cc:374] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:05.416141 11879 raft_consensus.cc:729] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8b27d6880d4ace9219e41348bc0e62, State: Initialized, Role: FOLLOWER
I20260812 00:39:05.416297 11877 raft_consensus.cc:348] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
I20260812 00:39:05.416298 11878 raft_consensus.cc:348] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
I20260812 00:39:05.416777 11877 raft_consensus.cc:374] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:05.416779 11878 raft_consensus.cc:374] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:05.416891 11877 raft_consensus.cc:729] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e4c21b44b314f818649a08592ca3521, State: Initialized, Role: FOLLOWER
I20260812 00:39:05.416895 11878 raft_consensus.cc:729] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d086567127114dcabaec6b9feabeda07, State: Initialized, Role: FOLLOWER
I20260812 00:39:05.416816 11879 consensus_queue.cc:260] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
I20260812 00:39:05.417591 11877 consensus_queue.cc:260] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
I20260812 00:39:05.417569 11878 consensus_queue.cc:260] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
I20260812 00:39:05.418223 11845 heartbeater.cc:503] Master 127.7.244.126:44111 was elected leader, sending a full tablet report...
I20260812 00:39:05.425021 11586 heartbeater.cc:503] Master 127.7.244.126:44111 was elected leader, sending a full tablet report...
I20260812 00:39:05.426311 11877 ts_tablet_manager.cc:1428] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Time spent starting tablet: real 0.021s user 0.010s sys 0.009s
I20260812 00:39:05.428210 11878 ts_tablet_manager.cc:1428] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Time spent starting tablet: real 0.023s user 0.020s sys 0.005s
I20260812 00:39:05.429061 11879 ts_tablet_manager.cc:1428] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Time spent starting tablet: real 0.022s user 0.006s sys 0.007s
W20260812 00:39:05.449609 11587 tablet.cc:2369] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:39:05.532519 11846 tablet.cc:2369] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:05.616744 11894 raft_consensus.cc:482] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:39:05.617112 11894 raft_consensus.cc:504] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
W20260812 00:39:05.617973 11327 tablet.cc:2369] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:05.619347 11894 leader_election.cc:288] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d086567127114dcabaec6b9feabeda07 (127.7.244.65:34495), 8e4c21b44b314f818649a08592ca3521 (127.7.244.66:43457)
I20260812 00:39:05.626521 11848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.619686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.7.244.67:48357)
0812 00:39:05.619895 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.619926 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:05.620064 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:05.620738 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:05.620743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:05.620775 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:05.621010 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.621024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.622381 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.622385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.624332 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.624350 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.625798 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.625803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.625955 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.625983 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 00:39:05.626063 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 00:39:05.626172 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":1}
I20260812 00:39:05.626528 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.619853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:48357 (local address 127.7.244.65:34495)
0812 00:39:05.620038 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.620043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.620067 (+ 24us) server_negotiation.cc:407] Connection header received
0812 00:39:05.620509 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.620514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.620596 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.620682 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:05.621148 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.622179 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.624519 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.625619 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.625800 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.625883 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.626124 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.626270 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:39:05.627249 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.620332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:52569 (local address 127.7.244.66:43457)
0812 00:39:05.620526 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.620532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.620865 (+ 333us) server_negotiation.cc:407] Connection header received
0812 00:39:05.621023 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.621027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.621092 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.621183 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:05.621881 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.622700 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.624968 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.626149 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.626303 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.626372 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.626921 (+ 549us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.627058 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:39:05.627396 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.620108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:43457 (local address 127.7.244.67:52569)
0812 00:39:05.620751 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.620768 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:05.620889 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:05.621214 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:05.621217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:05.621240 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:05.621690 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.621707 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.622894 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.622900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.624808 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:05.624821 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.626388 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.626393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:05.626737 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.626778 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 00:39:05.626870 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 00:39:05.627028 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":37,"thread_start_us":325,"threads_started":1}
I20260812 00:39:05.627887 11239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759409b9c2654ea1b02eaaf5c32265a0" candidate_uuid: "4d8b27d6880d4ace9219e41348bc0e62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d086567127114dcabaec6b9feabeda07" is_pre_election: true
I20260812 00:39:05.628453 11239 raft_consensus.cc:2454] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8b27d6880d4ace9219e41348bc0e62 in term 0.
I20260812 00:39:05.628485 11499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759409b9c2654ea1b02eaaf5c32265a0" candidate_uuid: "4d8b27d6880d4ace9219e41348bc0e62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e4c21b44b314f818649a08592ca3521" is_pre_election: true
I20260812 00:39:05.629035 11499 raft_consensus.cc:2454] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8b27d6880d4ace9219e41348bc0e62 in term 0.
I20260812 00:39:05.629146 11639 leader_election.cc:302] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8b27d6880d4ace9219e41348bc0e62, d086567127114dcabaec6b9feabeda07; no voters:
I20260812 00:39:05.629613 11894 raft_consensus.cc:2781] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:39:05.629772 11894 raft_consensus.cc:482] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:39:05.629896 11894 raft_consensus.cc:3037] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:05.632216 11894 raft_consensus.cc:504] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
I20260812 00:39:05.632977 11894 leader_election.cc:288] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [CANDIDATE]: Term 1 election: Requested vote from peers d086567127114dcabaec6b9feabeda07 (127.7.244.65:34495), 8e4c21b44b314f818649a08592ca3521 (127.7.244.66:43457)
I20260812 00:39:05.633656 11239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759409b9c2654ea1b02eaaf5c32265a0" candidate_uuid: "4d8b27d6880d4ace9219e41348bc0e62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d086567127114dcabaec6b9feabeda07"
I20260812 00:39:05.633653 11499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "759409b9c2654ea1b02eaaf5c32265a0" candidate_uuid: "4d8b27d6880d4ace9219e41348bc0e62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e4c21b44b314f818649a08592ca3521"
I20260812 00:39:05.633879 11239 raft_consensus.cc:3037] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:05.633903 11499 raft_consensus.cc:3037] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:05.636307 11239 raft_consensus.cc:2454] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8b27d6880d4ace9219e41348bc0e62 in term 1.
I20260812 00:39:05.636459 11499 raft_consensus.cc:2454] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8b27d6880d4ace9219e41348bc0e62 in term 1.
I20260812 00:39:05.636861 11639 leader_election.cc:302] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8b27d6880d4ace9219e41348bc0e62, d086567127114dcabaec6b9feabeda07; no voters:
I20260812 00:39:05.637202 11894 raft_consensus.cc:2781] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:05.637735 11894 raft_consensus.cc:686] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 1 LEADER]: Becoming Leader. State: Replica: 4d8b27d6880d4ace9219e41348bc0e62, State: Running, Role: LEADER
I20260812 00:39:05.638259 11894 consensus_queue.cc:237] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } }
I20260812 00:39:05.643991 10957 catalog_manager.cc:5495] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d8b27d6880d4ace9219e41348bc0e62 (127.7.244.67). New cstate: current_term: 1 leader_uuid: "4d8b27d6880d4ace9219e41348bc0e62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d8b27d6880d4ace9219e41348bc0e62" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 40577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 } health_report { overall_health: UNKNOWN } } }
I20260812 00:39:05.675565 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.669423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.7.244.67:40577)
0812 00:39:05.669640 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.669645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.669664 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:39:05.669851 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.669855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.669941 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.670073 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:39:05.670469 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.671212 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.672959 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.673478 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.673598 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.674322 (+ 724us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:39:05.674888 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:39:05.675027 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.675210 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.675299 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 00:39:05.684650 11239 raft_consensus.cc:1261] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 1 FOLLOWER]: Refusing update from remote peer 4d8b27d6880d4ace9219e41348bc0e62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:05.684653 11499 raft_consensus.cc:1261] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 1 FOLLOWER]: Refusing update from remote peer 4d8b27d6880d4ace9219e41348bc0e62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:05.685815 11901 consensus_queue.cc:1035] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d086567127114dcabaec6b9feabeda07" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:05.686239 11894 consensus_queue.cc:1035] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e4c21b44b314f818649a08592ca3521" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:05.697779 11907 mvcc.cc:205] Tried to move back new op lower bound from 7317484116712812544 to 7317484116538507264. Current Snapshot: MvccSnapshot[applied={T|T < 7317484116712812544}]
I20260812 00:39:05.702821 11911 mvcc.cc:205] Tried to move back new op lower bound from 7317484116712812544 to 7317484116538507264. Current Snapshot: MvccSnapshot[applied={T|T < 7317484116712812544}]
I20260812 00:39:05.705888 11908 mvcc.cc:205] Tried to move back new op lower bound from 7317484116712812544 to 7317484116538507264. Current Snapshot: MvccSnapshot[applied={T|T < 7317484116712812544}]
I20260812 00:39:05.721042 10956 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:05.721395 10956 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:05.725440 10956 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45bce2caab8745f0a83fe12beda9c146: Sending DeleteTablet for 3 replicas of tablet 759409b9c2654ea1b02eaaf5c32265a0
I20260812 00:39:05.726812 11468 tablet_service.cc:1514] Processing DeleteTablet for tablet 759409b9c2654ea1b02eaaf5c32265a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:05 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:47194
I20260812 00:39:05.726876 11202 tablet_service.cc:1514] Processing DeleteTablet for tablet 759409b9c2654ea1b02eaaf5c32265a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:05 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:34714
I20260812 00:39:05.726855 11724 tablet_service.cc:1514] Processing DeleteTablet for tablet 759409b9c2654ea1b02eaaf5c32265a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:05 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:38932
I20260812 00:39:05.727730 11923 tablet_replica.cc:330] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: stopping tablet replica
I20260812 00:39:05.728288 11923 raft_consensus.cc:2229] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:05.728981 11923 raft_consensus.cc:2258] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:05.729333 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11087
I20260812 00:39:05.734253 11923 ts_tablet_manager.cc:1905] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:05.742030 11923 ts_tablet_manager.cc:1918] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:05.742420 11923 log.cc:1196] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/759409b9c2654ea1b02eaaf5c32265a0
I20260812 00:39:05.742758 11924 tablet_replica.cc:330] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: stopping tablet replica
I20260812 00:39:05.743211 11923 ts_tablet_manager.cc:1939] T 759409b9c2654ea1b02eaaf5c32265a0 P 8e4c21b44b314f818649a08592ca3521: Deleting consensus metadata
I20260812 00:39:05.743284 11924 raft_consensus.cc:2229] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:05.743913 11924 raft_consensus.cc:2258] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 12 00:39:05 dist-test-slave-gjw7 krb5kdc[10872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:39:05.745654 10928 catalog_manager.cc:4841] TS 8e4c21b44b314f818649a08592ca3521 (127.7.244.66:43457): tablet 759409b9c2654ea1b02eaaf5c32265a0 (table test-table [id=5488ccd758804639870304530fba5cc7]) successfully deleted
I20260812 00:39:05.746672 11924 ts_tablet_manager.cc:1905] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:05.747943 11927 tablet_replica.cc:330] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: stopping tablet replica
I20260812 00:39:05.748430 11927 raft_consensus.cc:2229] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:39:05.749155 11927 raft_consensus.cc:2258] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:05.751931 11927 ts_tablet_manager.cc:1905] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:05.753461 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.730645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.7.244.65:34495)
0812 00:39:05.730832 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.730838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.732698 (+ 1860us) server_negotiation.cc:407] Connection header received
0812 00:39:05.733066 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.733073 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.733173 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.733265 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:05.735343 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.736548 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.738945 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.739641 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.739765 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.745675 (+ 5910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:05.745704 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:05.745725 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:05.745782 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:05.749115 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:05.750197 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:05.750210 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:05.750213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:05.750434 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:05.750996 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:05.751007 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:05.751010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:05.751577 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:05.751915 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.752668 (+ 753us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.752975 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:39:05.754583 11924 ts_tablet_manager.cc:1918] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:05.754776 11924 log.cc:1196] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/759409b9c2654ea1b02eaaf5c32265a0
I20260812 00:39:05.755102 11320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:05.760205 11927 ts_tablet_manager.cc:1918] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:05.760399 11927 log.cc:1196] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/759409b9c2654ea1b02eaaf5c32265a0
I20260812 00:39:05.760936 11927 ts_tablet_manager.cc:1939] T 759409b9c2654ea1b02eaaf5c32265a0 P 4d8b27d6880d4ace9219e41348bc0e62: Deleting consensus metadata
I20260812 00:39:05.762781 10917 catalog_manager.cc:4841] TS 4d8b27d6880d4ace9219e41348bc0e62 (127.7.244.67:40577): tablet 759409b9c2654ea1b02eaaf5c32265a0 (table test-table [id=5488ccd758804639870304530fba5cc7]) successfully deleted
I20260812 00:39:05.869325 11924 ts_tablet_manager.cc:1939] T 759409b9c2654ea1b02eaaf5c32265a0 P d086567127114dcabaec6b9feabeda07: Deleting consensus metadata
I20260812 00:39:05.869978 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 11087
W20260812 00:39:05.899689 10928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.244.65:34495 (error 108)
I20260812 00:39:05.899900 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11337
I20260812 00:39:05.902446 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.901554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34728)
0812 00:39:05.902069 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.902215 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":262,"threads_started":1}
W20260812 00:39:05.902765 10928 catalog_manager.cc:4584] TS d086567127114dcabaec6b9feabeda07 (127.7.244.65:34495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 759409b9c2654ea1b02eaaf5c32265a0: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Aug 12 00:39:05 dist-test-slave-gjw7 krb5kdc[10872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:39:05.917845 11876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.901086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.7.244.66:43457)
0812 00:39:05.901259 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:05.901264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:05.901282 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:39:05.901374 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:05.901380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:05.901462 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:05.901565 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:05.903400 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.904160 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.905668 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:05.906086 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:05.906202 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:05.911233 (+ 5031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:05.911253 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:05.911272 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:05.911317 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:05.914578 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:05.915616 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:05.915624 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:05.915627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:05.915774 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:05.916215 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:05.916218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:05.916220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:05.916671 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:05.916939 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:05.917355 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 00:39:05.917555 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 00:39:05.918757 11573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:05.956629 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.956014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34736)
0812 00:39:05.956305 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.956432 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260812 00:39:05.958060 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:05.957570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34738)
0812 00:39:05.957758 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:05.957859 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 00:39:06.009235 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 11337
I20260812 00:39:06.032473 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:06.031842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34742)
0812 00:39:06.032149 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:06.032274 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 00:39:06.033984 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:06.033420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34758)
0812 00:39:06.033682 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:06.033787 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260812 00:39:06.049537 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11593
Aug 12 00:39:06 dist-test-slave-gjw7 krb5kdc[10872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:39:06.068811 11872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:06.050560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.7.244.67:40577)
0812 00:39:06.050767 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:06.050773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:06.050793 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:39:06.050853 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:06.050857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:06.050929 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:06.051036 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:06.052425 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:06.053272 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:06.054840 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:06.055326 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:06.055462 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:06.060510 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:06.060535 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:06.060560 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:06.060626 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:06.065036 (+ 4410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:06.066175 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:06.066188 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:06.066191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:06.066451 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:06.066848 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:06.066853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:06.066857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:06.067457 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:06.067805 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:06.068120 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 00:39:06.068376 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 00:39:06.069739 11838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:06.114519 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:06.113964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34762)
0812 00:39:06.114240 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:06.114348 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260812 00:39:06.115753 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:06.115314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34766)
0812 00:39:06.115514 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:06.115593 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 00:39:06.194626 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 11593
I20260812 00:39:06.231601 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 10892
I20260812 00:39:06.232935 11017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:06.346196 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:06.345616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:34495 (local address 127.0.0.1:34782)
0812 00:39:06.345854 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:06.346003 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 00:39:06.346412 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 10892
2026-08-12T00:39:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5090 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T00:39:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:06Z Disabled control of system clock
I20260812 00:39:06.446448 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44169
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:44169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:39:06.815654 11959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:06.816017 11959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:06.816105 11959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:06.823904 11959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:06.824055 11959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:06.824095 11959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:06.824148 11959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:06.824193 11959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:06.834059 11959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:44169
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:44169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:06.835489 11959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:06.837675 11959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:06.850512 11969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:06.850567 11968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:06.850749 11977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:06.851472 11959 server_base.cc:1034] running on GCE node
I20260812 00:39:06.852362 11959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:06.854118 11959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:06.855362 11959 hybrid_clock.cc:648] HybridClock initialized: now 1786495146855304 us; error 51 us; skew 500 ppm
I20260812 00:39:06.855813 11959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:06.858537 11959 webserver.cc:460] Webserver started at http://127.7.244.126:45685/ using document root <none> and password file <none>
I20260812 00:39:06.859344 11959 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:06.859452 11959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:06.859807 11959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:06.863201 11959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "20a2c5266d6441d7a6946d481d20be3d"
format_stamp: "Formatted at 2026-08-12 00:39:06 on dist-test-slave-gjw7"
server_key: "bd8ea7e02182e3492d22e34ec8efbd17"
server_key_iv: "ec7adecb533b451fc0080603a1e09eec"
server_key_version: "encryptionkey@0"
I20260812 00:39:06.864224 11959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "20a2c5266d6441d7a6946d481d20be3d"
format_stamp: "Formatted at 2026-08-12 00:39:06 on dist-test-slave-gjw7"
server_key: "bd8ea7e02182e3492d22e34ec8efbd17"
server_key_iv: "ec7adecb533b451fc0080603a1e09eec"
server_key_version: "encryptionkey@0"
I20260812 00:39:06.870424 11959 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:39:06.874511 11995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:06.876238 11959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:39:06.876478 11959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "20a2c5266d6441d7a6946d481d20be3d"
format_stamp: "Formatted at 2026-08-12 00:39:06 on dist-test-slave-gjw7"
server_key: "bd8ea7e02182e3492d22e34ec8efbd17"
server_key_iv: "ec7adecb533b451fc0080603a1e09eec"
server_key_version: "encryptionkey@0"
I20260812 00:39:06.876747 11959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:06.901412 11959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:06.902454 11959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:06.902808 11959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:06.922655 11959 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:44169
I20260812 00:39:06.922677 12106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:44169 every 8 connection(s)
I20260812 00:39:06.924221 11959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:06.929177 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 11959
I20260812 00:39:06.929425 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:06.929915 12107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:06.930002 8145 external_mini_cluster.cc:1428] Setting key 97a48dca0ba8c9630708c964e2c5973d
I20260812 00:39:06.946169 12107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d: Bootstrap starting.
I20260812 00:39:06.948470 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:06.934918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.7.244.126:44169)
0812 00:39:06.936265 (+ 1347us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:06.936283 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:06.936345 (+ 62us) server_negotiation.cc:407] Connection header received
0812 00:39:06.938683 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:06.938718 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:06.939341 (+ 623us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:06.939838 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:06.940533 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:06.941978 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:06.943499 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:06.944312 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:06.944520 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:06.945842 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:06.945852 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:06.945869 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:06.946507 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:06.946605 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:06.946629 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:06.946740 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:06.946985 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 00:39:06.947348 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":1255,"threads_started":1}
I20260812 00:39:06.950070 12107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:06.951660 12107 log.cc:824] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:06.955195 12107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d: No bootstrap required, opened a new log
I20260812 00:39:06.964262 12107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a2c5266d6441d7a6946d481d20be3d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44169 } }
I20260812 00:39:06.964917 12107 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:06.965039 12107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20a2c5266d6441d7a6946d481d20be3d, State: Initialized, Role: FOLLOWER
I20260812 00:39:06.965788 12107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a2c5266d6441d7a6946d481d20be3d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44169 } }
I20260812 00:39:06.966084 12107 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:06.966235 12107 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:06.966400 12107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:06.969460 12107 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a2c5266d6441d7a6946d481d20be3d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44169 } }
I20260812 00:39:06.970114 12107 leader_election.cc:302] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20a2c5266d6441d7a6946d481d20be3d; no voters:
I20260812 00:39:06.970785 12107 leader_election.cc:288] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:06.970969 12115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:06.971880 12115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [term 1 LEADER]: Becoming Leader. State: Replica: 20a2c5266d6441d7a6946d481d20be3d, State: Running, Role: LEADER
I20260812 00:39:06.972491 12115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a2c5266d6441d7a6946d481d20be3d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44169 } }
I20260812 00:39:06.973153 12107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:06.975786 12120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20a2c5266d6441d7a6946d481d20be3d. Latest consensus state: current_term: 1 leader_uuid: "20a2c5266d6441d7a6946d481d20be3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a2c5266d6441d7a6946d481d20be3d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44169 } } }
I20260812 00:39:06.975989 12117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20a2c5266d6441d7a6946d481d20be3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a2c5266d6441d7a6946d481d20be3d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 44169 } } }
I20260812 00:39:06.976325 12120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:06.976346 12117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:06.979822 12132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:06.985603 12132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:06.995500 12132 catalog_manager.cc:1347] Generated new cluster ID: 33cdd63645b44bc7998baab16a6a1992
I20260812 00:39:06.995637 12132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:07.028770 12132 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:07.031078 12132 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:07.042646 12132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d: Generated new TSK 0
I20260812 00:39:07.043975 12132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:07.053953 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44169
--builtin_ntp_servers=127.7.244.84:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:39:07.559857 12152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:07.560199 12152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:07.560331 12152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:07.567927 12152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:07.568050 12152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:07.568207 12152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:39:07.577829 12152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:07.579267 12152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:07.581180 12152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:07.594960 12161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:07.594933 12162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:07.594993 12165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:07.596079 12152 server_base.cc:1034] running on GCE node
I20260812 00:39:07.596711 12152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:07.597777 12152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:07.598985 12152 hybrid_clock.cc:648] HybridClock initialized: now 1786495147598936 us; error 39 us; skew 500 ppm
I20260812 00:39:07.599308 12152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:07.601737 12152 webserver.cc:460] Webserver started at http://127.7.244.65:38661/ using document root <none> and password file <none>
I20260812 00:39:07.602456 12152 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:07.602553 12152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:07.602834 12152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:07.605938 12152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "7fef089ffccb43fab35115e9d1d4923d"
format_stamp: "Formatted at 2026-08-12 00:39:07 on dist-test-slave-gjw7"
server_key: "ae516496940f7ddb1036305629a66365"
server_key_iv: "9b2e6fcab3edb53cafde4790f305bb4f"
server_key_version: "encryptionkey@0"
I20260812 00:39:07.606766 12152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "7fef089ffccb43fab35115e9d1d4923d"
format_stamp: "Formatted at 2026-08-12 00:39:07 on dist-test-slave-gjw7"
server_key: "ae516496940f7ddb1036305629a66365"
server_key_iv: "9b2e6fcab3edb53cafde4790f305bb4f"
server_key_version: "encryptionkey@0"
I20260812 00:39:07.612357 12152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 00:39:07.616379 12172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:07.617950 12152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 00:39:07.618136 12152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "7fef089ffccb43fab35115e9d1d4923d"
format_stamp: "Formatted at 2026-08-12 00:39:07 on dist-test-slave-gjw7"
server_key: "ae516496940f7ddb1036305629a66365"
server_key_iv: "9b2e6fcab3edb53cafde4790f305bb4f"
server_key_version: "encryptionkey@0"
I20260812 00:39:07.618373 12152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:07.638652 12152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:07.639604 12152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:07.639938 12152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:07.641013 12152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:07.642741 12152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:07.642845 12152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:07.642964 12152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:07.643035 12152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:07.676904 12152 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:40589
I20260812 00:39:07.676952 12392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:40589 every 8 connection(s)
I20260812 00:39:07.678391 12152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:39:07.678960 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 12152
I20260812 00:39:07.679128 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:39:07.679792 8145 external_mini_cluster.cc:1428] Setting key 847b4ebcbe2557f13a1c1a7c038c494f
I20260812 00:39:07.683383 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44169
--builtin_ntp_servers=127.7.244.84:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:39:07.693822 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:07.681203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:35689 (local address 127.7.244.126:44169)
0812 00:39:07.681745 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:07.681751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:07.683419 (+ 1668us) server_negotiation.cc:407] Connection header received
0812 00:39:07.686000 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:07.686004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:07.686083 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:07.686180 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:07.687725 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:07.688469 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:07.690748 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:07.691168 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:07.691274 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:07.692992 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:07.692998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:07.693001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:07.693023 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:07.693076 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:07.693088 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:07.693158 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:07.693461 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 00:39:07.693559 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":293,"threads_started":1}
I20260812 00:39:07.695349 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:07.682073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44169 (local address 127.7.244.65:35689)
0812 00:39:07.683234 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:07.683290 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:07.685673 (+ 2383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:07.686502 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:07.686515 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:07.686648 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:07.687507 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:07.687535 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:07.688656 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:07.688661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:07.690592 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:07.690611 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:07.691443 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:07.691451 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:07.691749 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:07.692559 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:07.692562 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:07.692622 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:07.692750 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:07.693186 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:07.693199 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:07.693208 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 00:39:07.693455 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 00:39:07.693907 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":320,"threads_started":1}
I20260812 00:39:07.697235 12393 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44169
I20260812 00:39:07.697646 12393 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:07.698519 12393 heartbeater.cc:511] Master 127.7.244.126:44169 requested a full tablet report, sending...
I20260812 00:39:07.700894 12027 ts_manager.cc:194] Registered new tserver with Master: 7fef089ffccb43fab35115e9d1d4923d (127.7.244.65:40589)
I20260812 00:39:07.702559 12027 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.65:35689
W20260812 00:39:08.046911 12402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:08.047251 12402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:08.047387 12402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:08.055363 12402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:08.055516 12402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:08.055670 12402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:39:08.065761 12402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:08.067225 12402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:08.069239 12402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:08.083856 12408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:08.083809 12410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:08.083860 12407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:08.084709 12402 server_base.cc:1034] running on GCE node
I20260812 00:39:08.085462 12402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:08.086572 12402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:08.087817 12402 hybrid_clock.cc:648] HybridClock initialized: now 1786495148087761 us; error 47 us; skew 500 ppm
I20260812 00:39:08.088173 12402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:08.090615 12402 webserver.cc:460] Webserver started at http://127.7.244.66:34657/ using document root <none> and password file <none>
I20260812 00:39:08.091362 12402 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:08.091473 12402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:08.091796 12402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:08.094929 12402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "b28f902647ee45bb98cb112956684b5a"
format_stamp: "Formatted at 2026-08-12 00:39:08 on dist-test-slave-gjw7"
server_key: "88d3db8a09bd3bf34ef8ac91d7dac201"
server_key_iv: "5381ab86a36f73748c314e311afca433"
server_key_version: "encryptionkey@0"
I20260812 00:39:08.095899 12402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "b28f902647ee45bb98cb112956684b5a"
format_stamp: "Formatted at 2026-08-12 00:39:08 on dist-test-slave-gjw7"
server_key: "88d3db8a09bd3bf34ef8ac91d7dac201"
server_key_iv: "5381ab86a36f73748c314e311afca433"
server_key_version: "encryptionkey@0"
I20260812 00:39:08.101924 12402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 00:39:08.106415 12422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:08.108174 12402 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 00:39:08.108422 12402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "b28f902647ee45bb98cb112956684b5a"
format_stamp: "Formatted at 2026-08-12 00:39:08 on dist-test-slave-gjw7"
server_key: "88d3db8a09bd3bf34ef8ac91d7dac201"
server_key_iv: "5381ab86a36f73748c314e311afca433"
server_key_version: "encryptionkey@0"
I20260812 00:39:08.108688 12402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:08.133044 12402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:08.134068 12402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:08.134405 12402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:08.135532 12402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:08.137259 12402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:08.137408 12402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 00:39:08.137521 12402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:08.137591 12402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:08.171794 12402 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:43285
I20260812 00:39:08.171876 12665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:43285 every 8 connection(s)
I20260812 00:39:08.173210 12402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:39:08.178201 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 12402
I20260812 00:39:08.178417 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:39:08.179004 8145 external_mini_cluster.cc:1428] Setting key a2f9f1a0239711d964d286bbfdf0e82b
I20260812 00:39:08.182873 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44169
--builtin_ntp_servers=127.7.244.84:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:39:08.189618 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.176291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:45717 (local address 127.7.244.126:44169)
0812 00:39:08.176556 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.176562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.178512 (+ 1950us) server_negotiation.cc:407] Connection header received
0812 00:39:08.181080 (+ 2568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.181086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.181171 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.181308 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:08.183216 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.183984 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.186453 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.186910 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.187035 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.188799 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:08.188805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:08.188808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:08.188832 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:08.188866 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:08.188877 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:08.188951 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.189218 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.189343 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260812 00:39:08.191248 12668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.176905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44169 (local address 127.7.244.66:45717)
0812 00:39:08.178258 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:08.178329 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:08.180741 (+ 2412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:08.181772 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:08.181787 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:08.181948 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:08.182968 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.183004 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.184196 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.184203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.186202 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.186233 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.187182 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.187189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.187516 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.188375 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:08.188379 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:08.188444 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:08.188568 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:08.188975 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:08.188986 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:08.188995 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 00:39:08.189194 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 00:39:08.189724 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":435,"threads_started":1}
I20260812 00:39:08.192929 12666 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44169
I20260812 00:39:08.193343 12666 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:08.194352 12666 heartbeater.cc:511] Master 127.7.244.126:44169 requested a full tablet report, sending...
I20260812 00:39:08.196403 12027 ts_manager.cc:194] Registered new tserver with Master: b28f902647ee45bb98cb112956684b5a (127.7.244.66:43285)
I20260812 00:39:08.197582 12027 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.66:45717
W20260812 00:39:08.551059 12672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:08.551424 12672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:08.551519 12672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:08.559137 12672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:08.559273 12672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:08.559396 12672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:39:08.569072 12672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:44169
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:08.570525 12672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:08.572669 12672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:08.587842 12686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:08.587941 12683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:08.587787 12691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:08.589247 12672 server_base.cc:1034] running on GCE node
I20260812 00:39:08.590070 12672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:08.591300 12672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:08.592571 12672 hybrid_clock.cc:648] HybridClock initialized: now 1786495148592512 us; error 44 us; skew 500 ppm
I20260812 00:39:08.592980 12672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:08.599160 12672 webserver.cc:460] Webserver started at http://127.7.244.67:38975/ using document root <none> and password file <none>
I20260812 00:39:08.600023 12672 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:08.600170 12672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:08.600523 12672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:08.603803 12672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "f6d67092671843078617b102d39a9d86"
format_stamp: "Formatted at 2026-08-12 00:39:08 on dist-test-slave-gjw7"
server_key: "64dbacca65e8eac61194e542914259f3"
server_key_iv: "ad0cb9a156f4b37767bebe64c459e71b"
server_key_version: "encryptionkey@0"
I20260812 00:39:08.604679 12672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "f6d67092671843078617b102d39a9d86"
format_stamp: "Formatted at 2026-08-12 00:39:08 on dist-test-slave-gjw7"
server_key: "64dbacca65e8eac61194e542914259f3"
server_key_iv: "ad0cb9a156f4b37767bebe64c459e71b"
server_key_version: "encryptionkey@0"
I20260812 00:39:08.610631 12672 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260812 00:39:08.614822 12697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:08.616541 12672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260812 00:39:08.616739 12672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "f6d67092671843078617b102d39a9d86"
format_stamp: "Formatted at 2026-08-12 00:39:08 on dist-test-slave-gjw7"
server_key: "64dbacca65e8eac61194e542914259f3"
server_key_iv: "ad0cb9a156f4b37767bebe64c459e71b"
server_key_version: "encryptionkey@0"
I20260812 00:39:08.616997 12672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:08.663625 12672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:08.664799 12672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:08.665155 12672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:08.666349 12672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:08.668239 12672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:08.668373 12672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:08.668506 12672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:08.668584 12672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:08.703830 12672 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:44179
I20260812 00:39:08.703960 12906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:44179 every 8 connection(s)
I20260812 00:39:08.705335 12672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:39:08.706161 12393 heartbeater.cc:503] Master 127.7.244.126:44169 was elected leader, sending a full tablet report...
I20260812 00:39:08.708187 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 12672
I20260812 00:39:08.708410 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:39:08.708889 8145 external_mini_cluster.cc:1428] Setting key 4ef186e04fc2c0ec3bbecf68bb6873d9
I20260812 00:39:08.721068 12911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.709525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:47337 (local address 127.7.244.126:44169)
0812 00:39:08.713731 (+ 4206us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.713735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.713754 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:39:08.713807 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.713811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.713876 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.713977 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:08.715302 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.716125 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.718077 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.718517 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.718623 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.720176 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:08.720181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:08.720184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:08.720206 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:08.720246 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:08.720258 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:08.720339 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.720649 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.720761 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4080,"thread_start_us":3963,"threads_started":1}
I20260812 00:39:08.722321 12910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.709124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:44169 (local address 127.7.244.67:47337)
0812 00:39:08.710487 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:08.710547 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:08.713370 (+ 2823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:08.714292 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:08.714303 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:08.714415 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:08.715095 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.715115 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.716361 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.716366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.717888 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.717906 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.718745 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.718751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.718991 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.719817 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:08.719820 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:08.719868 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:08.719966 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:08.720412 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:08.720423 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:08.720431 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:39:08.720614 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 00:39:08.721019 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":418,"threads_started":1}
I20260812 00:39:08.723846 12907 heartbeater.cc:347] Connected to a master server at 127.7.244.126:44169
I20260812 00:39:08.724174 12907 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:08.725134 12907 heartbeater.cc:511] Master 127.7.244.126:44169 requested a full tablet report, sending...
I20260812 00:39:08.726608 12026 ts_manager.cc:194] Registered new tserver with Master: f6d67092671843078617b102d39a9d86 (127.7.244.67:44179)
I20260812 00:39:08.727553 12026 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.67:47337
I20260812 00:39:08.738097 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:39:08.749228 12911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.742779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36732 (local address 127.7.244.126:44169)
0812 00:39:08.743003 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.743008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.743400 (+ 392us) server_negotiation.cc:407] Connection header received
0812 00:39:08.743531 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.743536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.743626 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.743745 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:08.744368 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.745354 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.746841 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.747328 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.747439 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.748562 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:08.748576 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:08.748579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:08.748603 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:08.748643 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:08.748656 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:08.748747 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.748847 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.748955 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 00:39:08.753960 12026 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:39:08.761476 12026 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:39:08.802700 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.776308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:44179 (local address 127.0.0.1:33850)
0812 00:39:08.792329 (+ 16021us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:08.792366 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:08.792539 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:08.793864 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:08.793871 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:08.793899 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:08.794232 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.794243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.796351 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.796357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.799682 (+ 3325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.799701 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.801872 (+ 2171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.801879 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.802042 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.802075 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 00:39:08.802163 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 00:39:08.802278 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15864,"mutex_wait_us":196,"thread_start_us":16825,"threads_started":1}
I20260812 00:39:08.802726 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.776814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59874)
0812 00:39:08.792888 (+ 16074us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:08.792914 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:08.793504 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:08.794699 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:08.794704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:08.794732 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:08.795077 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.795087 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.796792 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.796798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.799710 (+ 2912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.799729 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.801421 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.801428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.801697 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.801762 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 00:39:08.801932 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 00:39:08.802187 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15892,"mutex_wait_us":141,"thread_start_us":16788,"threads_started":1}
I20260812 00:39:08.803494 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.776248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:43285 (local address 127.0.0.1:52406)
0812 00:39:08.783484 (+ 7236us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:08.783546 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:08.783791 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:08.795505 (+ 11714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:08.795510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:08.795538 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:08.795934 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.795949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.797813 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.797820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.800148 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.800163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.802242 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.802248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.802874 (+ 626us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.802914 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 00:39:08.802995 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 00:39:08.803124 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6887,"mutex_wait_us":3,"thread_start_us":6714,"threads_started":1}
I20260812 00:39:08.803771 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.777381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33850 (local address 127.7.244.67:44179)
0812 00:39:08.791806 (+ 14425us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.791815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.792716 (+ 901us) server_negotiation.cc:407] Connection header received
0812 00:39:08.792861 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.792871 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.793190 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.793482 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:08.794454 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.796149 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.799906 (+ 3757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.801356 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.801500 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.801784 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.803240 (+ 1456us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.803402 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14190,"thread_start_us":13589,"threads_started":1}
I20260812 00:39:08.802618 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.777163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.7.244.65:40589)
0812 00:39:08.790550 (+ 13387us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.790561 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.793089 (+ 2528us) server_negotiation.cc:407] Connection header received
0812 00:39:08.793759 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.793770 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.794054 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.794276 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:08.795254 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.796651 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.799986 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.801185 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.801362 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.801714 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.802006 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.802207 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13127,"thread_start_us":13403,"threads_started":1}
I20260812 00:39:08.804927 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.776563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52406 (local address 127.7.244.66:43285)
0812 00:39:08.794850 (+ 18287us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.794861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.794895 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:39:08.795000 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.795008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.795267 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.795488 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:08.796137 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.797644 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.800428 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.801984 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.803768 (+ 1784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.804096 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.804198 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.804423 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18085,"thread_start_us":17938,"threads_started":1}
I20260812 00:39:08.807699 12793 tablet_service.cc:1467] Processing CreateTablet for tablet 3b774bd09dbe4befae6a660faf00ab43 (DEFAULT_TABLE table=test-table [id=1391749d7ce64ec2860d53841df0bd24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:08.807699 12273 tablet_service.cc:1467] Processing CreateTablet for tablet 3b774bd09dbe4befae6a660faf00ab43 (DEFAULT_TABLE table=test-table [id=1391749d7ce64ec2860d53841df0bd24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:08.809221 12273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b774bd09dbe4befae6a660faf00ab43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:08.809340 12793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b774bd09dbe4befae6a660faf00ab43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:08.812687 12523 tablet_service.cc:1467] Processing CreateTablet for tablet 3b774bd09dbe4befae6a660faf00ab43 (DEFAULT_TABLE table=test-table [id=1391749d7ce64ec2860d53841df0bd24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:08.814468 12523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b774bd09dbe4befae6a660faf00ab43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:08.823109 12943 tablet_bootstrap.cc:492] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Bootstrap starting.
I20260812 00:39:08.826155 12944 tablet_bootstrap.cc:492] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: Bootstrap starting.
I20260812 00:39:08.828454 12943 tablet_bootstrap.cc:654] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:08.829975 12944 tablet_bootstrap.cc:654] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:08.829991 12943 log.cc:824] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:08.830237 12945 tablet_bootstrap.cc:492] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Bootstrap starting.
I20260812 00:39:08.831339 12944 log.cc:824] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:08.834136 12943 tablet_bootstrap.cc:492] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: No bootstrap required, opened a new log
I20260812 00:39:08.834151 12945 tablet_bootstrap.cc:654] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:08.834442 12943 ts_tablet_manager.cc:1397] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.001s
I20260812 00:39:08.835544 12945 log.cc:824] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:08.836481 12944 tablet_bootstrap.cc:492] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: No bootstrap required, opened a new log
I20260812 00:39:08.836783 12944 ts_tablet_manager.cc:1397] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260812 00:39:08.839342 12945 tablet_bootstrap.cc:492] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: No bootstrap required, opened a new log
I20260812 00:39:08.839715 12945 ts_tablet_manager.cc:1397] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260812 00:39:08.842895 12943 raft_consensus.cc:348] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.843322 12943 raft_consensus.cc:374] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:08.843425 12943 raft_consensus.cc:729] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fef089ffccb43fab35115e9d1d4923d, State: Initialized, Role: FOLLOWER
I20260812 00:39:08.844096 12943 consensus_queue.cc:260] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.846104 12943 ts_tablet_manager.cc:1428] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Time spent starting tablet: real 0.012s user 0.005s sys 0.007s
I20260812 00:39:08.847702 12944 raft_consensus.cc:348] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.848225 12944 raft_consensus.cc:374] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:08.848337 12944 raft_consensus.cc:729] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6d67092671843078617b102d39a9d86, State: Initialized, Role: FOLLOWER
I20260812 00:39:08.849530 12944 consensus_queue.cc:260] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.850759 12945 raft_consensus.cc:348] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.851271 12945 raft_consensus.cc:374] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:08.851374 12945 raft_consensus.cc:729] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b28f902647ee45bb98cb112956684b5a, State: Initialized, Role: FOLLOWER
I20260812 00:39:08.851625 12944 ts_tablet_manager.cc:1428] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260812 00:39:08.852200 12945 consensus_queue.cc:260] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.852768 12907 heartbeater.cc:503] Master 127.7.244.126:44169 was elected leader, sending a full tablet report...
I20260812 00:39:08.853432 12666 heartbeater.cc:503] Master 127.7.244.126:44169 was elected leader, sending a full tablet report...
I20260812 00:39:08.854224 12945 ts_tablet_manager.cc:1428] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Time spent starting tablet: real 0.014s user 0.004s sys 0.009s
I20260812 00:39:08.865523 12953 raft_consensus.cc:482] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:39:08.865854 12953 raft_consensus.cc:504] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.867682 12953 leader_election.cc:288] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b28f902647ee45bb98cb112956684b5a (127.7.244.66:43285), 7fef089ffccb43fab35115e9d1d4923d (127.7.244.65:40589)
I20260812 00:39:08.873917 12910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.867909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:43285 (local address 127.7.244.67:39849)
0812 00:39:08.868369 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:08.868390 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:08.868518 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:08.869341 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:08.869345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:08.869374 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:08.869531 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.869539 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.870534 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.870538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.872002 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.872017 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.873147 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.873152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.873354 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.873391 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 00:39:08.873480 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 00:39:08.873611 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319}
I20260812 00:39:08.873982 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.868768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:39849 (local address 127.7.244.66:43285)
0812 00:39:08.868969 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.868974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.868994 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:39:08.869065 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.869070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.869157 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.869239 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:08.869683 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.870396 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.872199 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.872984 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.873137 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.873212 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.873570 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.873722 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 00:39:08.874650 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.868287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:60695 (local address 127.7.244.65:40589)
0812 00:39:08.868506 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.868514 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.869108 (+ 594us) server_negotiation.cc:407] Connection header received
0812 00:39:08.869412 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.869417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.869492 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.869572 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:08.870136 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.870843 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.872681 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.873637 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.873792 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.873862 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.874237 (+ 375us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.874382 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 00:39:08.875308 12958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.868129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.7.244.67:60695)
0812 00:39:08.869003 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:08.869018 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:08.869226 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:08.869611 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:08.869616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:08.869640 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:08.869956 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.869966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.871010 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.871014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.872463 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:08.872477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.873826 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.873832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:08.874019 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.874063 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 00:39:08.874849 (+ 786us) client_negotiation.cc:240] Negotiation successful
0812 00:39:08.875002 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":535,"threads_started":1}
I20260812 00:39:08.875236 12569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b774bd09dbe4befae6a660faf00ab43" candidate_uuid: "f6d67092671843078617b102d39a9d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b28f902647ee45bb98cb112956684b5a" is_pre_election: true
I20260812 00:39:08.875859 12569 raft_consensus.cc:2454] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6d67092671843078617b102d39a9d86 in term 0.
I20260812 00:39:08.876389 12309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b774bd09dbe4befae6a660faf00ab43" candidate_uuid: "f6d67092671843078617b102d39a9d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fef089ffccb43fab35115e9d1d4923d" is_pre_election: true
I20260812 00:39:08.876559 12699 leader_election.cc:302] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b28f902647ee45bb98cb112956684b5a, f6d67092671843078617b102d39a9d86; no voters:
I20260812 00:39:08.876899 12309 raft_consensus.cc:2454] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6d67092671843078617b102d39a9d86 in term 0.
I20260812 00:39:08.876946 12953 raft_consensus.cc:2781] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:39:08.877072 12953 raft_consensus.cc:482] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:39:08.877167 12953 raft_consensus.cc:3037] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:08.879736 12953 raft_consensus.cc:504] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.880517 12953 leader_election.cc:288] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [CANDIDATE]: Term 1 election: Requested vote from peers b28f902647ee45bb98cb112956684b5a (127.7.244.66:43285), 7fef089ffccb43fab35115e9d1d4923d (127.7.244.65:40589)
I20260812 00:39:08.881096 12569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b774bd09dbe4befae6a660faf00ab43" candidate_uuid: "f6d67092671843078617b102d39a9d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b28f902647ee45bb98cb112956684b5a"
I20260812 00:39:08.881148 12309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b774bd09dbe4befae6a660faf00ab43" candidate_uuid: "f6d67092671843078617b102d39a9d86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fef089ffccb43fab35115e9d1d4923d"
I20260812 00:39:08.881341 12569 raft_consensus.cc:3037] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:08.881393 12309 raft_consensus.cc:3037] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:08.883776 12309 raft_consensus.cc:2454] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6d67092671843078617b102d39a9d86 in term 1.
I20260812 00:39:08.883924 12569 raft_consensus.cc:2454] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6d67092671843078617b102d39a9d86 in term 1.
I20260812 00:39:08.884364 12704 leader_election.cc:302] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fef089ffccb43fab35115e9d1d4923d, f6d67092671843078617b102d39a9d86; no voters:
I20260812 00:39:08.884723 12953 raft_consensus.cc:2781] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:08.885179 12953 raft_consensus.cc:686] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 1 LEADER]: Becoming Leader. State: Replica: f6d67092671843078617b102d39a9d86, State: Running, Role: LEADER
I20260812 00:39:08.885706 12953 consensus_queue.cc:237] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } }
I20260812 00:39:08.890496 12027 catalog_manager.cc:5495] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6d67092671843078617b102d39a9d86 (127.7.244.67). New cstate: current_term: 1 leader_uuid: "f6d67092671843078617b102d39a9d86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6d67092671843078617b102d39a9d86" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 } health_report { overall_health: UNKNOWN } } }
W20260812 00:39:08.925555 12667 tablet.cc:2369] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:08.927877 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.920985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33862 (local address 127.7.244.67:44179)
0812 00:39:08.921202 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.921208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.921231 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:39:08.921552 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.921557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.921656 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.921811 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:39:08.922266 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.923090 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.925376 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.926000 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.926142 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:08.926777 (+ 635us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:39:08.927313 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:39:08.927451 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:08.927589 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 00:39:08.927666 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 00:39:08.931154 12395 tablet.cc:2369] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:08.937809 12309 raft_consensus.cc:1261] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 1 FOLLOWER]: Refusing update from remote peer f6d67092671843078617b102d39a9d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:08.937808 12569 raft_consensus.cc:1261] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 1 FOLLOWER]: Refusing update from remote peer f6d67092671843078617b102d39a9d86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:08.938941 12953 consensus_queue.cc:1035] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fef089ffccb43fab35115e9d1d4923d" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 40589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:08.939880 12964 consensus_queue.cc:1035] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b28f902647ee45bb98cb112956684b5a" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:08.950300 12974 mvcc.cc:205] Tried to move back new op lower bound from 7317484130037436416 to 7317484129839808512. Current Snapshot: MvccSnapshot[applied={T|T < 7317484130037436416}]
I20260812 00:39:08.953051 12975 mvcc.cc:205] Tried to move back new op lower bound from 7317484130037436416 to 7317484129839808512. Current Snapshot: MvccSnapshot[applied={T|T < 7317484130037436416}]
I20260812 00:39:08.969553 12026 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:08.969866 12026 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:08.973990 12026 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20a2c5266d6441d7a6946d481d20be3d: Sending DeleteTablet for 3 replicas of tablet 3b774bd09dbe4befae6a660faf00ab43
I20260812 00:39:08.975369 12273 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b774bd09dbe4befae6a660faf00ab43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:08 UTC) from {username='slave'} at 127.0.0.1:59874
I20260812 00:39:08.975369 12523 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b774bd09dbe4befae6a660faf00ab43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:08 UTC) from {username='slave'} at 127.0.0.1:52406
I20260812 00:39:08.976131 12793 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b774bd09dbe4befae6a660faf00ab43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:08 UTC) from {username='slave'} at 127.0.0.1:33850
I20260812 00:39:08.977370 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 12152
I20260812 00:39:08.988511 12985 tablet_replica.cc:330] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: stopping tablet replica
I20260812 00:39:08.990023 12985 raft_consensus.cc:2229] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:08.991497 12985 raft_consensus.cc:2258] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:08.995471 12985 ts_tablet_manager.cc:1905] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:08.996744 12989 tablet_replica.cc:330] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: stopping tablet replica
I20260812 00:39:08.997406 12989 raft_consensus.cc:2229] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:08.998103 12989 raft_consensus.cc:2258] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:09.000211 12991 tablet_replica.cc:330] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: stopping tablet replica
I20260812 00:39:09.000702 12991 raft_consensus.cc:2229] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:39:09.003091 12991 raft_consensus.cc:2258] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:09.003631 12989 ts_tablet_manager.cc:1905] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:09.004104 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:08.979357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59876 (local address 127.7.244.65:40589)
0812 00:39:08.979575 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:08.979582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:08.993216 (+ 13634us) server_negotiation.cc:407] Connection header received
0812 00:39:08.993805 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:08.993812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:08.993950 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:08.994034 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:08.994754 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.995858 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.998034 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:08.998640 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:08.998753 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:09.001362 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:09.001375 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:09.001396 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:09.002250 (+ 854us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:09.002330 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:09.002357 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:09.002493 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:09.003500 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0812 00:39:09.003641 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 00:39:09.005434 12385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:09.008908 12991 ts_tablet_manager.cc:1905] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:09.012070 12989 ts_tablet_manager.cc:1918] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:09.012429 12989 log.cc:1196] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/3b774bd09dbe4befae6a660faf00ab43
I20260812 00:39:09.013309 12989 ts_tablet_manager.cc:1939] T 3b774bd09dbe4befae6a660faf00ab43 P b28f902647ee45bb98cb112956684b5a: Deleting consensus metadata
I20260812 00:39:09.015794 11996 catalog_manager.cc:4841] TS b28f902647ee45bb98cb112956684b5a (127.7.244.66:43285): tablet 3b774bd09dbe4befae6a660faf00ab43 (table test-table [id=1391749d7ce64ec2860d53841df0bd24]) successfully deleted
I20260812 00:39:09.018107 12991 ts_tablet_manager.cc:1918] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:09.018373 12991 log.cc:1196] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/3b774bd09dbe4befae6a660faf00ab43
I20260812 00:39:09.019140 12991 ts_tablet_manager.cc:1939] T 3b774bd09dbe4befae6a660faf00ab43 P f6d67092671843078617b102d39a9d86: Deleting consensus metadata
I20260812 00:39:09.021371 11999 catalog_manager.cc:4841] TS f6d67092671843078617b102d39a9d86 (127.7.244.67:44179): tablet 3b774bd09dbe4befae6a660faf00ab43 (table test-table [id=1391749d7ce64ec2860d53841df0bd24]) successfully deleted
I20260812 00:39:09.124655 12985 ts_tablet_manager.cc:1918] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:09.124939 12985 log.cc:1196] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/3b774bd09dbe4befae6a660faf00ab43
I20260812 00:39:09.125475 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 12152
I20260812 00:39:09.125572 12985 ts_tablet_manager.cc:1939] T 3b774bd09dbe4befae6a660faf00ab43 P 7fef089ffccb43fab35115e9d1d4923d: Deleting consensus metadata
W20260812 00:39:09.157779 11998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.244.65:40589 (error 108)
I20260812 00:39:09.158659 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 12402
I20260812 00:39:09.160148 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.159547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59890)
0812 00:39:09.159822 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:09.159960 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260812 00:39:09.160461 11998 catalog_manager.cc:4584] TS 7fef089ffccb43fab35115e9d1d4923d (127.7.244.65:40589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b774bd09dbe4befae6a660faf00ab43: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
I20260812 00:39:09.167546 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.159657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52422 (local address 127.7.244.66:43285)
0812 00:39:09.159839 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:09.159845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:09.160116 (+ 271us) server_negotiation.cc:407] Connection header received
0812 00:39:09.160201 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:09.160206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:09.160339 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:09.160451 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:09.160996 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:09.162201 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:09.163608 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:09.164324 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:09.164478 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:09.165766 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:09.165778 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:09.165803 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:09.166651 (+ 848us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:09.166722 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:09.166750 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:09.166879 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:09.167029 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 00:39:09.167165 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:39:09.168351 12658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:09.214551 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.213949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59906)
0812 00:39:09.214222 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:09.214340 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 00:39:09.215947 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.215431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59918)
0812 00:39:09.215622 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:09.215731 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:39:09.259176 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 12402
I20260812 00:39:09.276768 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.276177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59930)
0812 00:39:09.276445 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:09.276556 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 00:39:09.278079 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.277634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59936)
0812 00:39:09.277814 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:09.277915 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 00:39:09.289206 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 12672
I20260812 00:39:09.298262 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.290101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33868 (local address 127.7.244.67:44179)
0812 00:39:09.290332 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:09.290338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:09.290362 (+ 24us) server_negotiation.cc:407] Connection header received
0812 00:39:09.290512 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:09.290518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:09.290627 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:09.290760 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:09.291119 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:09.292383 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:09.294056 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:09.294744 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:09.294913 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:09.296280 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:09.296299 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:09.296332 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:09.297232 (+ 900us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:09.297363 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:09.297398 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:09.297536 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:09.297735 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 00:39:09.297871 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 00:39:09.299197 12901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:09.359328 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.358748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59952)
0812 00:39:09.359041 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:09.359155 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260812 00:39:09.360643 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.360123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59968)
0812 00:39:09.360355 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:09.360509 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260812 00:39:09.423040 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 12672
I20260812 00:39:09.454938 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 11959
I20260812 00:39:09.456236 12092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:09.570562 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:09.569745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:40589 (local address 127.0.0.1:59978)
0812 00:39:09.570152 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:09.570312 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260812 00:39:09.570736 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 11959
2026-08-12T00:39:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3224 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:09 dist-test-slave-gjw7 krb5kdc[13010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:09 dist-test-slave-gjw7 krb5kdc[13010](info): set up 2 sockets
Aug 12 00:39:09 dist-test-slave-gjw7 krb5kdc[13010](info): commencing operation
krb5kdc: starting...
W20260812 00:39:10.904896 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.007s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:11 dist-test-slave-gjw7 krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:39:11.218444 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33861
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:33861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:11.587019 13028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:11.587378 13028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:11.587525 13028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:11.595211 13028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:11.595357 13028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:11.595441 13028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:11.595520 13028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:11.595580 13028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:11.606040 13028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:33861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:11.607470 13028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:11.608878 13028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:11.621460 13035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:11.621464 13040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:11.621551 13036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:11.622972 13028 server_base.cc:1034] running on GCE node
I20260812 00:39:11.623925 13028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:11.625684 13028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:11.626966 13028 hybrid_clock.cc:648] HybridClock initialized: now 1786495151626898 us; error 47 us; skew 500 ppm
I20260812 00:39:11.627416 13028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:11 dist-test-slave-gjw7 krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495151, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:11.631073 13028 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:39:11.632963 13028 webserver.cc:460] Webserver started at http://127.7.244.126:37223/ using document root <none> and password file <none>
I20260812 00:39:11.633816 13028 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:11.633951 13028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:11.634297 13028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:11.637821 13028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "f3f50f0094294437aa79efa453799cea"
format_stamp: "Formatted at 2026-08-12 00:39:11 on dist-test-slave-gjw7"
server_key: "ba7cf509bc74b6b8a4704929f81d3a07"
server_key_iv: "3d0bf8bce61b68cd42cd648c4347929f"
server_key_version: "encryptionkey@0"
I20260812 00:39:11.638736 13028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "f3f50f0094294437aa79efa453799cea"
format_stamp: "Formatted at 2026-08-12 00:39:11 on dist-test-slave-gjw7"
server_key: "ba7cf509bc74b6b8a4704929f81d3a07"
server_key_iv: "3d0bf8bce61b68cd42cd648c4347929f"
server_key_version: "encryptionkey@0"
I20260812 00:39:11.644894 13028 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:39:11.649335 13052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:11.651029 13028 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:39:11.651238 13028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "f3f50f0094294437aa79efa453799cea"
format_stamp: "Formatted at 2026-08-12 00:39:11 on dist-test-slave-gjw7"
server_key: "ba7cf509bc74b6b8a4704929f81d3a07"
server_key_iv: "3d0bf8bce61b68cd42cd648c4347929f"
server_key_version: "encryptionkey@0"
I20260812 00:39:11.651479 13028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:11.665421 13028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:11.670166 13028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:11.670468 13028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:11.690200 13028 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:33861
I20260812 00:39:11.690208 13146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:33861 every 8 connection(s)
I20260812 00:39:11.691715 13028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:11.696110 13151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:11.700317 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 13028
I20260812 00:39:11.700528 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:11.701143 8145 external_mini_cluster.cc:1428] Setting key 9056df23965e9c928e5a6303d237102d
I20260812 00:39:11.710347 13151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea: Bootstrap starting.
I20260812 00:39:11.714510 13151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:11.716006 13151 log.cc:824] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:39:11 dist-test-slave-gjw7 krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:11.719939 13151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea: No bootstrap required, opened a new log
I20260812 00:39:11.727552 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:11.703294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.7.244.126:33861)
0812 00:39:11.704176 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:11.704189 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:11.704238 (+ 49us) server_negotiation.cc:407] Connection header received
0812 00:39:11.706030 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:11.706057 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:11.706521 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:11.706893 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:11.708748 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:11.710170 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:11.712048 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:11.712752 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:11.712882 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:11.718143 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:11.718156 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:11.718178 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:11.718234 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:11.722323 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:11.723206 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:11.723215 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:11.723229 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:11.723488 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:11.723843 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:11.723852 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:11.723856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:11.724678 (+ 822us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:11.724969 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:11.725516 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 00:39:11.726055 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":398,"threads_started":1}
I20260812 00:39:11.730577 13151 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f50f0094294437aa79efa453799cea" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33861 } }
I20260812 00:39:11.731071 13151 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:11.731186 13151 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3f50f0094294437aa79efa453799cea, State: Initialized, Role: FOLLOWER
I20260812 00:39:11.731855 13151 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f50f0094294437aa79efa453799cea" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33861 } }
I20260812 00:39:11.732124 13151 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:11.732262 13151 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:11.732431 13151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:11.734815 13151 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f50f0094294437aa79efa453799cea" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33861 } }
I20260812 00:39:11.735447 13151 leader_election.cc:302] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3f50f0094294437aa79efa453799cea; no voters:
I20260812 00:39:11.736171 13151 leader_election.cc:288] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:11.736281 13159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:11.737414 13159 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [term 1 LEADER]: Becoming Leader. State: Replica: f3f50f0094294437aa79efa453799cea, State: Running, Role: LEADER
I20260812 00:39:11.738005 13159 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f50f0094294437aa79efa453799cea" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33861 } }
I20260812 00:39:11.738440 13151 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:11.740968 13163 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3f50f0094294437aa79efa453799cea. Latest consensus state: current_term: 1 leader_uuid: "f3f50f0094294437aa79efa453799cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f50f0094294437aa79efa453799cea" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33861 } } }
I20260812 00:39:11.741513 13163 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:11.742020 13160 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3f50f0094294437aa79efa453799cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3f50f0094294437aa79efa453799cea" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33861 } } }
I20260812 00:39:11.742331 13160 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:11.744875 13179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:11.749778 13179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:11.759431 13179 catalog_manager.cc:1347] Generated new cluster ID: ac387680a9ce4d54b5e0e508738eca28
I20260812 00:39:11.759562 13179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:11.773159 13179 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:11.774627 13179 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:11.786160 13179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea: Generated new TSK 0
I20260812 00:39:11.787177 13179 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:39:11.873327 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33861
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:12.244210 13203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:12.244549 13203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:12.244676 13203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:12.252250 13203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:12.252401 13203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:12.252548 13203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:39:12.263835 13203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:12.265316 13203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:12.266760 13203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:12.281160 13217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:12.281075 13218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:12.281833 13221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:12.281905 13203 server_base.cc:1034] running on GCE node
I20260812 00:39:12.282612 13203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:12.283653 13203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:12.284897 13203 hybrid_clock.cc:648] HybridClock initialized: now 1786495152284835 us; error 53 us; skew 500 ppm
I20260812 00:39:12.285336 13203 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:12 dist-test-slave-gjw7 krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495152, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:12.288970 13203 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:39:12.290654 13203 webserver.cc:460] Webserver started at http://127.7.244.65:33725/ using document root <none> and password file <none>
I20260812 00:39:12.291435 13203 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:12.291558 13203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:12.291893 13203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:12.295171 13203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "643146e97000416fb4c1180c23433a46"
format_stamp: "Formatted at 2026-08-12 00:39:12 on dist-test-slave-gjw7"
server_key: "2144ff38b21a3b608c6b985b2af04ebc"
server_key_iv: "7a36d91b3f3ef8d5310488b216f11a13"
server_key_version: "encryptionkey@0"
I20260812 00:39:12.296135 13203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "643146e97000416fb4c1180c23433a46"
format_stamp: "Formatted at 2026-08-12 00:39:12 on dist-test-slave-gjw7"
server_key: "2144ff38b21a3b608c6b985b2af04ebc"
server_key_iv: "7a36d91b3f3ef8d5310488b216f11a13"
server_key_version: "encryptionkey@0"
I20260812 00:39:12.302035 13203 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 00:39:12.306447 13231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:12.308074 13203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:39:12.308277 13203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "643146e97000416fb4c1180c23433a46"
format_stamp: "Formatted at 2026-08-12 00:39:12 on dist-test-slave-gjw7"
server_key: "2144ff38b21a3b608c6b985b2af04ebc"
server_key_iv: "7a36d91b3f3ef8d5310488b216f11a13"
server_key_version: "encryptionkey@0"
I20260812 00:39:12.308502 13203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:12.328344 13203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:12.333173 13203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:12.333523 13203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:12.334676 13203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:12.336361 13203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:12.336453 13203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:12.336589 13203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:12.336645 13203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:12.369515 13203 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:41695
I20260812 00:39:12.369552 13446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:41695 every 8 connection(s)
I20260812 00:39:12.370891 13203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:39:12.377125 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 13203
I20260812 00:39:12.377307 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:39:12.377869 8145 external_mini_cluster.cc:1428] Setting key 0b6ed5129830114aa641b27100da6496
Aug 12 00:39:12 dist-test-slave-gjw7 krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495152, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:12.395421 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:12.373633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:49533 (local address 127.7.244.126:33861)
0812 00:39:12.374108 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:12.374113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:12.375013 (+ 900us) server_negotiation.cc:407] Connection header received
0812 00:39:12.377772 (+ 2759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:12.377777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:12.377863 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:12.377996 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:12.380138 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:12.380909 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:12.383365 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:12.383822 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:12.383945 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:12.388698 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:12.388704 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:12.388707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:12.388748 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:12.391559 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:12.392637 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:12.392641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:12.392644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:12.392780 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:12.393345 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:12.393349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:12.393351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:12.393682 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:12.393846 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:12.394758 (+ 912us) server_negotiation.cc:299] Negotiation successful
0812 00:39:12.395034 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":228,"threads_started":1}
I20260812 00:39:12.396525 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:12.374072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33861 (local address 127.7.244.65:49533)
0812 00:39:12.374828 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:12.374870 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:12.376639 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:12.378329 (+ 1690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:12.378343 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:12.379116 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:12.379913 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:12.379949 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:12.381101 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:12.381106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:12.383168 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:12.383186 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:12.384129 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:12.384137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:12.384462 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:12.385322 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:12.385367 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:12.388340 (+ 2973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:12.391777 (+ 3437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:12.391789 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:12.391809 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:12.392501 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:12.392898 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:12.392903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:12.392906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:12.393150 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:12.393838 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:12.393848 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:12.394261 (+ 413us) client_negotiation.cc:769] Sending connection context
0812 00:39:12.394574 (+ 313us) client_negotiation.cc:240] Negotiation successful
0812 00:39:12.395212 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":291,"threads_started":1}
I20260812 00:39:12.398545 13447 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33861
I20260812 00:39:12.398891 13447 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:12.399725 13447 heartbeater.cc:511] Master 127.7.244.126:33861 requested a full tablet report, sending...
I20260812 00:39:12.402458 13079 ts_manager.cc:194] Registered new tserver with Master: 643146e97000416fb4c1180c23433a46 (127.7.244.65:41695)
I20260812 00:39:12.404529 13079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:49533
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:39:12.449676 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33861
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:12.802937 13460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:12.803263 13460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:12.803390 13460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:12.811025 13460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:12.811162 13460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:12.811308 13460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:39:12.822080 13460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:12.823486 13460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:12.824991 13460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:12.839346 13475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:12.839375 13479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:12.839402 13472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:12.840835 13460 server_base.cc:1034] running on GCE node
I20260812 00:39:12.841588 13460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:12.842741 13460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:12.843977 13460 hybrid_clock.cc:648] HybridClock initialized: now 1786495152843922 us; error 55 us; skew 500 ppm
I20260812 00:39:12.844334 13460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:12 dist-test-slave-gjw7 krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495152, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:12.847792 13460 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:39:12.849466 13460 webserver.cc:460] Webserver started at http://127.7.244.66:32829/ using document root <none> and password file <none>
I20260812 00:39:12.850216 13460 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:12.850325 13460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:12.850621 13460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:12.853711 13460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "7dff117bf6aa457584d6c1bf25937632"
format_stamp: "Formatted at 2026-08-12 00:39:12 on dist-test-slave-gjw7"
server_key: "5e9f4d6fd359a815b74dd95a4a228a69"
server_key_iv: "ec3b9c6b7797d0a1e45ce87f4fc47502"
server_key_version: "encryptionkey@0"
I20260812 00:39:12.854502 13460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "7dff117bf6aa457584d6c1bf25937632"
format_stamp: "Formatted at 2026-08-12 00:39:12 on dist-test-slave-gjw7"
server_key: "5e9f4d6fd359a815b74dd95a4a228a69"
server_key_iv: "ec3b9c6b7797d0a1e45ce87f4fc47502"
server_key_version: "encryptionkey@0"
I20260812 00:39:12.860181 13460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:39:12.864028 13497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:12.865727 13460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 00:39:12.865916 13460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "7dff117bf6aa457584d6c1bf25937632"
format_stamp: "Formatted at 2026-08-12 00:39:12 on dist-test-slave-gjw7"
server_key: "5e9f4d6fd359a815b74dd95a4a228a69"
server_key_iv: "ec3b9c6b7797d0a1e45ce87f4fc47502"
server_key_version: "encryptionkey@0"
I20260812 00:39:12.866123 13460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:12.886574 13460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:12.891610 13460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:12.891893 13460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:12.892937 13460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:12.894496 13460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:12.894575 13460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:12.894668 13460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:12.894712 13460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:12.927059 13460 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:35637
I20260812 00:39:12.927099 13721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:35637 every 8 connection(s)
I20260812 00:39:12.928469 13460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:39:12.931794 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 13460
I20260812 00:39:12.931975 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:39:12.932605 8145 external_mini_cluster.cc:1428] Setting key 74b56745f973823f9d67f3706008a043
Aug 12 00:39:12 dist-test-slave-gjw7 krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495152, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:12.953143 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:12.931263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:46601 (local address 127.7.244.126:33861)
0812 00:39:12.931824 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:12.931831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:12.932965 (+ 1134us) server_negotiation.cc:407] Connection header received
0812 00:39:12.935174 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:12.935179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:12.935246 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:12.935334 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:12.937657 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:12.938419 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:12.940737 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:12.941266 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:12.941409 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:12.946160 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:12.946165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:12.946168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:12.946209 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:12.949502 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:12.950445 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:12.950451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:12.950454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:12.950628 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:12.951117 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:12.951123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:12.951127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:12.951507 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:12.951660 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:12.952526 (+ 866us) server_negotiation.cc:299] Negotiation successful
0812 00:39:12.952774 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":288,"threads_started":1}
I20260812 00:39:12.954186 13725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:12.931789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33861 (local address 127.7.244.66:46601)
0812 00:39:12.932762 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:12.932819 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:12.934809 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:12.935641 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:12.935654 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:12.936505 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:12.937457 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:12.937489 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:12.938601 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:12.938608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:12.940575 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:12.940593 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:12.941552 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:12.941561 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:12.941940 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:12.942767 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:12.942810 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:12.945852 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:12.949697 (+ 3845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:12.949706 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:12.949723 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:12.950272 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:12.950751 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:12.950755 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:12.950757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:12.950990 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:12.951670 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:12.951678 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:12.952077 (+ 399us) client_negotiation.cc:769] Sending connection context
0812 00:39:12.952378 (+ 301us) client_negotiation.cc:240] Negotiation successful
0812 00:39:12.952826 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":349,"threads_started":1}
I20260812 00:39:12.955828 13722 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33861
I20260812 00:39:12.956286 13722 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:12.957341 13722 heartbeater.cc:511] Master 127.7.244.126:33861 requested a full tablet report, sending...
I20260812 00:39:12.959301 13079 ts_manager.cc:194] Registered new tserver with Master: 7dff117bf6aa457584d6c1bf25937632 (127.7.244.66:35637)
I20260812 00:39:12.960212 13079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:46601
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:39:13.005862 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33861
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:33447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:13.375878 13729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:13.376307 13729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:13.376477 13729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:13.384158 13729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:13.384301 13729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:13.384418 13729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:39:13.394761 13729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:33447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:13.396139 13729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:13.397645 13729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:39:13.408262 13447 heartbeater.cc:503] Master 127.7.244.126:33861 was elected leader, sending a full tablet report...
W20260812 00:39:13.412755 13738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:13.412745 13741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:13.412781 13745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:13.413122 13729 server_base.cc:1034] running on GCE node
I20260812 00:39:13.414212 13729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:13.415280 13729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:13.416513 13729 hybrid_clock.cc:648] HybridClock initialized: now 1786495153416449 us; error 52 us; skew 500 ppm
I20260812 00:39:13.416885 13729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:13 dist-test-slave-gjw7 krb5kdc[13010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495153, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:13.420624 13729 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:39:13.422403 13729 webserver.cc:460] Webserver started at http://127.7.244.67:38767/ using document root <none> and password file <none>
I20260812 00:39:13.423172 13729 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:13.423309 13729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:13.423647 13729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:13.427039 13729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "99ab52955af14c6a96bbd8c918a5caec"
format_stamp: "Formatted at 2026-08-12 00:39:13 on dist-test-slave-gjw7"
server_key: "12fdf38db4e167c95195e0f003556b42"
server_key_iv: "5535f8b2c39e50658a143186415273a8"
server_key_version: "encryptionkey@0"
I20260812 00:39:13.427907 13729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "99ab52955af14c6a96bbd8c918a5caec"
format_stamp: "Formatted at 2026-08-12 00:39:13 on dist-test-slave-gjw7"
server_key: "12fdf38db4e167c95195e0f003556b42"
server_key_iv: "5535f8b2c39e50658a143186415273a8"
server_key_version: "encryptionkey@0"
I20260812 00:39:13.433889 13729 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:39:13.438145 13759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:13.440004 13729 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 00:39:13.440201 13729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "99ab52955af14c6a96bbd8c918a5caec"
format_stamp: "Formatted at 2026-08-12 00:39:13 on dist-test-slave-gjw7"
server_key: "12fdf38db4e167c95195e0f003556b42"
server_key_iv: "5535f8b2c39e50658a143186415273a8"
server_key_version: "encryptionkey@0"
I20260812 00:39:13.440424 13729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:13.480106 13729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:13.485721 13729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:13.486044 13729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:13.487190 13729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:13.488715 13729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:13.488801 13729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:13.488910 13729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:13.488956 13729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:13.522171 13729 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:43299
I20260812 00:39:13.522221 13995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:43299 every 8 connection(s)
I20260812 00:39:13.523587 13729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:39:13.528577 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 13729
I20260812 00:39:13.528786 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:39:13.529435 8145 external_mini_cluster.cc:1428] Setting key 38d7d9a79ecb4de37bbfcada297f4168
Aug 12 00:39:13 dist-test-slave-gjw7 krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495153, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:13.547434 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.526407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:49557 (local address 127.7.244.126:33861)
0812 00:39:13.526952 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.526959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.528127 (+ 1168us) server_negotiation.cc:407] Connection header received
0812 00:39:13.530812 (+ 2685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.530816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.530905 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.531029 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:13.533422 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.534227 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.535961 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.536388 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.536501 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.540637 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:13.540643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:13.540646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:13.540690 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:13.543646 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:13.544793 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:13.544798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:13.544800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:13.544955 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:13.545557 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:13.545561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:13.545564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:13.545897 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:13.546071 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:13.546896 (+ 825us) server_negotiation.cc:299] Negotiation successful
0812 00:39:13.547116 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":246,"threads_started":1}
I20260812 00:39:13.548353 14005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.527144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33861 (local address 127.7.244.67:49557)
0812 00:39:13.527934 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:13.527986 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:13.530436 (+ 2450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:13.531382 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:13.531394 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:13.532358 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:13.533184 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.533211 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.534387 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.534391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.535757 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.535774 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.536670 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.536676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.536917 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.537696 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:13.537729 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:13.540320 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:13.543906 (+ 3586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:13.543919 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:13.543947 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:13.544654 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:13.545084 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:13.545089 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:13.545091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:13.545406 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:13.546027 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:13.546034 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:13.546477 (+ 443us) client_negotiation.cc:769] Sending connection context
0812 00:39:13.546735 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 00:39:13.547184 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":319,"threads_started":1}
I20260812 00:39:13.549872 13998 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33861
I20260812 00:39:13.550256 13998 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:13.551046 13998 heartbeater.cc:511] Master 127.7.244.126:33861 requested a full tablet report, sending...
I20260812 00:39:13.552541 13079 ts_manager.cc:194] Registered new tserver with Master: 99ab52955af14c6a96bbd8c918a5caec (127.7.244.67:43299)
I20260812 00:39:13.553396 13079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:49557
I20260812 00:39:13.559060 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:39:13.577868 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.563700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59064 (local address 127.7.244.126:33861)
0812 00:39:13.563902 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.563907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.564196 (+ 289us) server_negotiation.cc:407] Connection header received
0812 00:39:13.564434 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.564439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.564537 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.564634 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:13.566317 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.567066 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.568519 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.568988 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.569099 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.572200 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:13.572206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:13.572210 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:13.572255 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:13.574942 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:13.575791 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:13.575795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:13.575797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:13.575951 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:13.576402 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:13.576405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:13.576407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:13.576768 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:13.576980 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:13.577477 (+ 497us) server_negotiation.cc:299] Negotiation successful
0812 00:39:13.577641 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:39:13.582342 13079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:39:13.589498 13079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:39:13.615520 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.602973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:43299 (local address 127.0.0.1:35958)
0812 00:39:13.605146 (+ 2173us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:13.605162 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:13.605267 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:13.606373 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:13.606380 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:13.606425 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:13.606940 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.606960 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.609222 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.609229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.611710 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.611732 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.614341 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.614350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.614619 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.614679 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 00:39:13.614821 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 00:39:13.615046 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":1,"thread_start_us":560,"threads_started":1}
I20260812 00:39:13.617449 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.603270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.7.244.67:43299)
0812 00:39:13.604518 (+ 1248us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.604527 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.605425 (+ 898us) server_negotiation.cc:407] Connection header received
0812 00:39:13.605549 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.605557 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.605827 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.606039 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:13.607167 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.608993 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.611945 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.613752 (+ 1807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.613899 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.614184 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:13.616756 (+ 2572us) server_negotiation.cc:299] Negotiation successful
0812 00:39:13.616990 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1061,"thread_start_us":910,"threads_started":1}
I20260812 00:39:13.622663 13850 tablet_service.cc:1467] Processing CreateTablet for tablet f6558a12608643f7a7ade70d4739d2f4 (DEFAULT_TABLE table=test-table [id=11e0963fce4c468cb208f18fc1f74554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:13.624236 13850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6558a12608643f7a7ade70d4739d2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:13.626031 14023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.604765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:35637 (local address 127.0.0.1:44426)
0812 00:39:13.616417 (+ 11652us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:13.616440 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:13.616554 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:13.618429 (+ 1875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:13.618435 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:13.618464 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:13.618658 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.618673 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.621047 (+ 2374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.621053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.623044 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.623064 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.625034 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.625042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.625343 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.625381 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 00:39:13.625506 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 00:39:13.625679 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11515,"thread_start_us":20476,"threads_started":1}
I20260812 00:39:13.626123 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.605031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.7.244.66:35637)
0812 00:39:13.617420 (+ 12389us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.617432 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.617471 (+ 39us) server_negotiation.cc:407] Connection header received
0812 00:39:13.617594 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.617604 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.617931 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.618450 (+ 519us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:13.618855 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.620823 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.623314 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.624809 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.624982 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.625188 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:13.625602 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 00:39:13.625783 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12155,"thread_start_us":13082,"threads_started":1}
I20260812 00:39:13.626358 14025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.603824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:41695 (local address 127.0.0.1:59164)
0812 00:39:13.604675 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:13.604726 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:13.604942 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:13.618801 (+ 13859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:13.618805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:13.618836 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:13.619259 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.619274 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.621480 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.621485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.623534 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.623549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.625519 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.625527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.625732 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.625767 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:39:13.625834 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 00:39:13.625975 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":1,"thread_start_us":554,"threads_started":1}
I20260812 00:39:13.627465 14026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.604504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59164 (local address 127.7.244.65:41695)
0812 00:39:13.618191 (+ 13687us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.618199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.618228 (+ 29us) server_negotiation.cc:407] Connection header received
0812 00:39:13.618322 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.618330 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.618573 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.619504 (+ 931us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:13.619791 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.621318 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.623764 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.625193 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.626626 (+ 1433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.626877 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:13.626957 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 00:39:13.627138 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13526,"thread_start_us":13390,"threads_started":1}
I20260812 00:39:13.630438 13595 tablet_service.cc:1467] Processing CreateTablet for tablet f6558a12608643f7a7ade70d4739d2f4 (DEFAULT_TABLE table=test-table [id=11e0963fce4c468cb208f18fc1f74554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:13.631783 13595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6558a12608643f7a7ade70d4739d2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:13.631718 13320 tablet_service.cc:1467] Processing CreateTablet for tablet f6558a12608643f7a7ade70d4739d2f4 (DEFAULT_TABLE table=test-table [id=11e0963fce4c468cb208f18fc1f74554]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:13.633446 13320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6558a12608643f7a7ade70d4739d2f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:13.640765 14030 tablet_bootstrap.cc:492] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Bootstrap starting.
I20260812 00:39:13.645399 14030 tablet_bootstrap.cc:654] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:13.646392 14031 tablet_bootstrap.cc:492] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Bootstrap starting.
I20260812 00:39:13.647110 14030 log.cc:824] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:13.649930 14036 tablet_bootstrap.cc:492] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Bootstrap starting.
I20260812 00:39:13.650504 14030 tablet_bootstrap.cc:492] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: No bootstrap required, opened a new log
I20260812 00:39:13.650681 14031 tablet_bootstrap.cc:654] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:13.650863 14030 ts_tablet_manager.cc:1397] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260812 00:39:13.651834 14031 log.cc:824] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:13.654429 14036 tablet_bootstrap.cc:654] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:13.655021 14031 tablet_bootstrap.cc:492] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: No bootstrap required, opened a new log
I20260812 00:39:13.655792 14031 ts_tablet_manager.cc:1397] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 00:39:13.655900 14036 log.cc:824] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:13.660715 14036 tablet_bootstrap.cc:492] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: No bootstrap required, opened a new log
I20260812 00:39:13.661226 14036 ts_tablet_manager.cc:1397] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.001s
I20260812 00:39:13.662217 14030 raft_consensus.cc:348] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.662793 14030 raft_consensus.cc:374] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:13.662920 14030 raft_consensus.cc:729] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99ab52955af14c6a96bbd8c918a5caec, State: Initialized, Role: FOLLOWER
I20260812 00:39:13.663671 14030 consensus_queue.cc:260] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.664968 13998 heartbeater.cc:503] Master 127.7.244.126:33861 was elected leader, sending a full tablet report...
I20260812 00:39:13.665750 14030 ts_tablet_manager.cc:1428] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260812 00:39:13.666893 14031 raft_consensus.cc:348] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.667448 14031 raft_consensus.cc:374] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:13.667579 14031 raft_consensus.cc:729] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dff117bf6aa457584d6c1bf25937632, State: Initialized, Role: FOLLOWER
I20260812 00:39:13.668292 14031 consensus_queue.cc:260] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.671324 14036 raft_consensus.cc:348] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.671860 14036 raft_consensus.cc:374] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:13.672029 14036 raft_consensus.cc:729] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 643146e97000416fb4c1180c23433a46, State: Initialized, Role: FOLLOWER
I20260812 00:39:13.672765 14036 consensus_queue.cc:260] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.674043 13722 heartbeater.cc:503] Master 127.7.244.126:33861 was elected leader, sending a full tablet report...
I20260812 00:39:13.674891 14036 ts_tablet_manager.cc:1428] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Time spent starting tablet: real 0.013s user 0.006s sys 0.006s
I20260812 00:39:13.675958 14031 ts_tablet_manager.cc:1428] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Time spent starting tablet: real 0.019s user 0.003s sys 0.012s
W20260812 00:39:13.680505 13723 tablet.cc:2369] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:39:13.775368 14000 tablet.cc:2369] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:13.868396 14047 raft_consensus.cc:482] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:39:13.868737 14047 raft_consensus.cc:504] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.870759 14047 leader_election.cc:288] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99ab52955af14c6a96bbd8c918a5caec (127.7.244.67:43299), 7dff117bf6aa457584d6c1bf25937632 (127.7.244.66:35637)
W20260812 00:39:13.873663 13451 tablet.cc:2369] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:13.878023 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.871760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:52639 (local address 127.7.244.66:35637)
0812 00:39:13.871965 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.871972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.872035 (+ 63us) server_negotiation.cc:407] Connection header received
0812 00:39:13.872295 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.872299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.872364 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.872448 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:13.873092 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.874091 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.876108 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.876922 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.877102 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.877189 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:13.877591 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 00:39:13.877740 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 00:39:13.878078 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.871340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:51065 (local address 127.7.244.67:43299)
0812 00:39:13.871577 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.871584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.872581 (+ 997us) server_negotiation.cc:407] Connection header received
0812 00:39:13.872760 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.872764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.872831 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.872927 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:13.873535 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.874479 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.876262 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.877066 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.877230 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.877345 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:13.877715 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 00:39:13.877842 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 00:39:13.878151 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.871133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:35637 (local address 127.7.244.65:52639)
0812 00:39:13.871923 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:13.871946 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:13.872130 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:13.872491 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:13.872496 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:13.872524 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:13.872928 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.872939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.874245 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.874251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.875888 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.875904 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.877165 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.877172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.877400 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.877448 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 00:39:13.877542 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 00:39:13.877698 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"spinlock_wait_cycles":22400,"thread_start_us":376,"threads_started":1}
I20260812 00:39:13.878742 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.871410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:43299 (local address 127.7.244.65:51065)
0812 00:39:13.872447 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:13.872464 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:13.872617 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:13.872959 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:13.872963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:13.872988 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:13.873323 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.873337 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.874631 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.874635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.876082 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:13.876097 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.877332 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.877340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:13.877510 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.877545 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:39:13.878301 (+ 756us) client_negotiation.cc:240] Negotiation successful
0812 00:39:13.878430 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"thread_start_us":674,"threads_started":1}
I20260812 00:39:13.879549 13631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6558a12608643f7a7ade70d4739d2f4" candidate_uuid: "643146e97000416fb4c1180c23433a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dff117bf6aa457584d6c1bf25937632" is_pre_election: true
I20260812 00:39:13.879966 13894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6558a12608643f7a7ade70d4739d2f4" candidate_uuid: "643146e97000416fb4c1180c23433a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ab52955af14c6a96bbd8c918a5caec" is_pre_election: true
I20260812 00:39:13.880229 13631 raft_consensus.cc:2454] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643146e97000416fb4c1180c23433a46 in term 0.
I20260812 00:39:13.880515 13894 raft_consensus.cc:2454] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 643146e97000416fb4c1180c23433a46 in term 0.
I20260812 00:39:13.881220 13232 leader_election.cc:302] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 643146e97000416fb4c1180c23433a46, 7dff117bf6aa457584d6c1bf25937632, 99ab52955af14c6a96bbd8c918a5caec; no voters:
I20260812 00:39:13.881793 14047 raft_consensus.cc:2781] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:39:13.881990 14047 raft_consensus.cc:482] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:39:13.882110 14047 raft_consensus.cc:3037] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:13.885066 14047 raft_consensus.cc:504] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.885988 14047 leader_election.cc:288] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [CANDIDATE]: Term 1 election: Requested vote from peers 99ab52955af14c6a96bbd8c918a5caec (127.7.244.67:43299), 7dff117bf6aa457584d6c1bf25937632 (127.7.244.66:35637)
I20260812 00:39:13.886726 13631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6558a12608643f7a7ade70d4739d2f4" candidate_uuid: "643146e97000416fb4c1180c23433a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dff117bf6aa457584d6c1bf25937632"
I20260812 00:39:13.886972 13631 raft_consensus.cc:3037] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:13.887080 13894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6558a12608643f7a7ade70d4739d2f4" candidate_uuid: "643146e97000416fb4c1180c23433a46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99ab52955af14c6a96bbd8c918a5caec"
I20260812 00:39:13.887322 13894 raft_consensus.cc:3037] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:13.889669 13631 raft_consensus.cc:2454] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643146e97000416fb4c1180c23433a46 in term 1.
I20260812 00:39:13.889991 13894 raft_consensus.cc:2454] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 643146e97000416fb4c1180c23433a46 in term 1.
I20260812 00:39:13.890326 13232 leader_election.cc:302] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 643146e97000416fb4c1180c23433a46, 7dff117bf6aa457584d6c1bf25937632; no voters:
I20260812 00:39:13.890697 14047 raft_consensus.cc:2781] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:13.891188 14047 raft_consensus.cc:686] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 1 LEADER]: Becoming Leader. State: Replica: 643146e97000416fb4c1180c23433a46, State: Running, Role: LEADER
I20260812 00:39:13.891685 14047 consensus_queue.cc:237] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } }
I20260812 00:39:13.897420 13079 catalog_manager.cc:5495] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 reported cstate change: term changed from 0 to 1, leader changed from <none> to 643146e97000416fb4c1180c23433a46 (127.7.244.65). New cstate: current_term: 1 leader_uuid: "643146e97000416fb4c1180c23433a46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "643146e97000416fb4c1180c23433a46" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 41695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 } health_report { overall_health: UNKNOWN } } }
I20260812 00:39:13.915663 14026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.910091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59166 (local address 127.7.244.65:41695)
0812 00:39:13.910322 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.910326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.910347 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:39:13.910547 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.910551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.910636 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.910769 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:39:13.911119 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.911833 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.913389 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.913823 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.913948 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:13.914447 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:39:13.915001 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:39:13.915146 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:13.915328 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 00:39:13.915376 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:39:13.924566 13631 raft_consensus.cc:1261] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 1 FOLLOWER]: Refusing update from remote peer 643146e97000416fb4c1180c23433a46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:13.924566 13894 raft_consensus.cc:1261] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 1 FOLLOWER]: Refusing update from remote peer 643146e97000416fb4c1180c23433a46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:13.925659 14054 consensus_queue.cc:1035] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dff117bf6aa457584d6c1bf25937632" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:13.926036 14047 consensus_queue.cc:1035] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99ab52955af14c6a96bbd8c918a5caec" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 43299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:13.957585 13077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:13.957981 13077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:13.962885 13077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3f50f0094294437aa79efa453799cea: Sending DeleteTablet for 3 replicas of tablet f6558a12608643f7a7ade70d4739d2f4
I20260812 00:39:13.964501 13850 tablet_service.cc:1514] Processing DeleteTablet for tablet f6558a12608643f7a7ade70d4739d2f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:13 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:35958
I20260812 00:39:13.964495 13595 tablet_service.cc:1514] Processing DeleteTablet for tablet f6558a12608643f7a7ade70d4739d2f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:13 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:44426
I20260812 00:39:13.965394 13320 tablet_service.cc:1514] Processing DeleteTablet for tablet f6558a12608643f7a7ade70d4739d2f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:13 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:59164
I20260812 00:39:13.966363 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13203
I20260812 00:39:13.976982 14085 tablet_replica.cc:330] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: stopping tablet replica
I20260812 00:39:13.978840 14085 raft_consensus.cc:2229] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:39:13.979751 14085 raft_consensus.cc:2258] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:13.984447 14085 ts_tablet_manager.cc:1905] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:13.987661 14083 tablet_replica.cc:330] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: stopping tablet replica
I20260812 00:39:13.988389 14083 raft_consensus.cc:2229] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:13.991567 14083 raft_consensus.cc:2258] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:13.994688 14083 ts_tablet_manager.cc:1905] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:13.995393 14084 tablet_replica.cc:330] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: stopping tablet replica
I20260812 00:39:13.995678 14085 ts_tablet_manager.cc:1918] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:13.995805 14084 raft_consensus.cc:2229] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:13.995901 14085 log.cc:1196] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/f6558a12608643f7a7ade70d4739d2f4
I20260812 00:39:13.996322 14084 raft_consensus.cc:2258] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:13.996567 14085 ts_tablet_manager.cc:1939] T f6558a12608643f7a7ade70d4739d2f4 P 643146e97000416fb4c1180c23433a46: Deleting consensus metadata
I20260812 00:39:13.998639 13053 catalog_manager.cc:4841] TS 643146e97000416fb4c1180c23433a46 (127.7.244.65:41695): tablet f6558a12608643f7a7ade70d4739d2f4 (table test-table [id=11e0963fce4c468cb208f18fc1f74554]) successfully deleted
I20260812 00:39:13.998901 14084 ts_tablet_manager.cc:1905] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 00:39:13 dist-test-slave-gjw7 krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:39:14.006600 14084 ts_tablet_manager.cc:1918] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:14.006953 14084 log.cc:1196] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/f6558a12608643f7a7ade70d4739d2f4
I20260812 00:39:14.007355 14083 ts_tablet_manager.cc:1918] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:14.007515 14084 ts_tablet_manager.cc:1939] T f6558a12608643f7a7ade70d4739d2f4 P 99ab52955af14c6a96bbd8c918a5caec: Deleting consensus metadata
I20260812 00:39:14.007553 14083 log.cc:1196] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/f6558a12608643f7a7ade70d4739d2f4
I20260812 00:39:14.008136 14083 ts_tablet_manager.cc:1939] T f6558a12608643f7a7ade70d4739d2f4 P 7dff117bf6aa457584d6c1bf25937632: Deleting consensus metadata
I20260812 00:39:14.009680 13057 catalog_manager.cc:4841] TS 99ab52955af14c6a96bbd8c918a5caec (127.7.244.67:43299): tablet f6558a12608643f7a7ade70d4739d2f4 (table test-table [id=11e0963fce4c468cb208f18fc1f74554]) successfully deleted
I20260812 00:39:14.010018 13053 catalog_manager.cc:4841] TS 7dff117bf6aa457584d6c1bf25937632 (127.7.244.66:35637): tablet f6558a12608643f7a7ade70d4739d2f4 (table test-table [id=11e0963fce4c468cb208f18fc1f74554]) successfully deleted
I20260812 00:39:14.013216 14026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:13.967992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59170 (local address 127.7.244.65:41695)
0812 00:39:13.968249 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:13.968256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:13.980843 (+ 12587us) server_negotiation.cc:407] Connection header received
0812 00:39:13.981191 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:13.981198 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:13.981328 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:13.981412 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:13.983773 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.984856 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.988765 (+ 3909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:13.989731 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:13.989860 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:14.004619 (+ 14759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:14.004638 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:14.004662 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:14.004725 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:14.009322 (+ 4597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:14.010657 (+ 1335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:14.010665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:14.010668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:14.010854 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:14.011325 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:14.011330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:14.011332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:14.011808 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:14.012152 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:14.012544 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 00:39:14.012821 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 00:39:14.014000 13437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:14.116747 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 13203
I20260812 00:39:14.147564 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13460
Aug 12 00:39:14 dist-test-slave-gjw7 krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:39:14.164220 14029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:14.148499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.7.244.66:35637)
0812 00:39:14.148708 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:14.148714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:14.148731 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:39:14.148786 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:14.148790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:14.148874 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:14.148961 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:14.150402 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:14.151170 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:14.152533 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:14.152941 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:14.153062 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:14.157959 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:14.157979 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:14.157997 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:14.158042 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:14.161045 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:14.161987 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:14.161995 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:14.161997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:14.162144 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:14.162616 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:14.162620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:14.162622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:14.163057 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:14.163308 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:14.163703 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 00:39:14.163892 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 00:39:14.165052 13717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:14.255541 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 13460
I20260812 00:39:14.287333 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13729
Aug 12 00:39:14 dist-test-slave-gjw7 krb5kdc[13010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:39:14.303535 14024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:14.288197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.7.244.67:43299)
0812 00:39:14.288442 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:14.288448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:14.288471 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:39:14.288591 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:14.288595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:14.288685 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:14.288780 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:14.290385 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:14.291119 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:14.292517 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:14.292927 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:14.293049 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:14.297758 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:14.297777 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:14.297796 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:14.297840 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:14.300528 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:14.301350 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:14.301364 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:14.301366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:14.301521 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:14.301912 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:14.301917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:14.301919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:14.302319 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:14.302611 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:14.302953 (+ 342us) server_negotiation.cc:299] Negotiation successful
0812 00:39:14.303172 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:39:14.304205 13989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:14.393939 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 13729
I20260812 00:39:14.419931 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 13028
I20260812 00:39:14.421149 13138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:14.532900 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 13028
2026-08-12T00:39:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4965 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T00:39:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:14Z Disabled control of system clock
I20260812 00:39:14.638372 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:46505
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:46505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:39:15.005955 14104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:15.006289 14104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:15.006381 14104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:15.014056 14104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:15.014194 14104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:15.014231 14104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:15.014281 14104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:15.014325 14104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:15.024083 14104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:46505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:46505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:15.025516 14104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:15.027518 14104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:15.040200 14114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:15.040158 14115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:15.040128 14118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:15.041316 14104 server_base.cc:1034] running on GCE node
I20260812 00:39:15.042275 14104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:15.043911 14104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:15.045193 14104 hybrid_clock.cc:648] HybridClock initialized: now 1786495155045121 us; error 56 us; skew 500 ppm
I20260812 00:39:15.045665 14104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:15.048394 14104 webserver.cc:460] Webserver started at http://127.7.244.126:43245/ using document root <none> and password file <none>
I20260812 00:39:15.049191 14104 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:15.049357 14104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:15.049724 14104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:15.052956 14104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "19cd39281db54eaa8afee03ab266e879"
format_stamp: "Formatted at 2026-08-12 00:39:15 on dist-test-slave-gjw7"
server_key: "1177d3160482211efb9276288effd98d"
server_key_iv: "f533878d2f6b90abc91031f11196d1b7"
server_key_version: "encryptionkey@0"
I20260812 00:39:15.053866 14104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "19cd39281db54eaa8afee03ab266e879"
format_stamp: "Formatted at 2026-08-12 00:39:15 on dist-test-slave-gjw7"
server_key: "1177d3160482211efb9276288effd98d"
server_key_iv: "f533878d2f6b90abc91031f11196d1b7"
server_key_version: "encryptionkey@0"
I20260812 00:39:15.059398 14104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260812 00:39:15.063198 14130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:15.064672 14104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 00:39:15.064863 14104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "19cd39281db54eaa8afee03ab266e879"
format_stamp: "Formatted at 2026-08-12 00:39:15 on dist-test-slave-gjw7"
server_key: "1177d3160482211efb9276288effd98d"
server_key_iv: "f533878d2f6b90abc91031f11196d1b7"
server_key_version: "encryptionkey@0"
I20260812 00:39:15.065089 14104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:15.079476 14104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:15.080410 14104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:15.080716 14104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:15.099164 14104 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:46505
I20260812 00:39:15.099182 14227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:46505 every 8 connection(s)
I20260812 00:39:15.100680 14104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:15.104801 14228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:15.111065 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 14104
I20260812 00:39:15.111260 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:15.111712 8145 external_mini_cluster.cc:1428] Setting key 3b5df93c2ea80b34d1b85c02a4d5f3a7
I20260812 00:39:15.117329 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879: Bootstrap starting.
I20260812 00:39:15.121850 14228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:15.122997 14228 log.cc:824] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:15.126842 14228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879: No bootstrap required, opened a new log
I20260812 00:39:15.128754 14235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:15.113941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.7.244.126:46505)
0812 00:39:15.115155 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:15.115171 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:15.115230 (+ 59us) server_negotiation.cc:407] Connection header received
0812 00:39:15.117741 (+ 2511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:15.117778 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:15.118470 (+ 692us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:15.118967 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:15.119610 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:15.121645 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:15.123346 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:15.124286 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:15.124440 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:15.125862 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:15.125874 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:15.125896 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:15.126648 (+ 752us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:15.126772 (+ 124us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:15.126803 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:15.126931 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:15.127278 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 00:39:15.127639 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":362,"threads_started":1}
I20260812 00:39:15.135705 14228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19cd39281db54eaa8afee03ab266e879" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46505 } }
I20260812 00:39:15.136195 14228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:15.136307 14228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19cd39281db54eaa8afee03ab266e879, State: Initialized, Role: FOLLOWER
I20260812 00:39:15.136961 14228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19cd39281db54eaa8afee03ab266e879" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46505 } }
I20260812 00:39:15.137261 14228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:15.137485 14228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:15.137691 14228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:15.140041 14228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19cd39281db54eaa8afee03ab266e879" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46505 } }
I20260812 00:39:15.140591 14228 leader_election.cc:302] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19cd39281db54eaa8afee03ab266e879; no voters:
I20260812 00:39:15.141223 14228 leader_election.cc:288] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:15.141427 14241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:15.142189 14241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [term 1 LEADER]: Becoming Leader. State: Replica: 19cd39281db54eaa8afee03ab266e879, State: Running, Role: LEADER
I20260812 00:39:15.142768 14241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19cd39281db54eaa8afee03ab266e879" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46505 } }
I20260812 00:39:15.143563 14228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:15.146106 14243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19cd39281db54eaa8afee03ab266e879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19cd39281db54eaa8afee03ab266e879" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46505 } } }
I20260812 00:39:15.146620 14243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:15.148473 14244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19cd39281db54eaa8afee03ab266e879. Latest consensus state: current_term: 1 leader_uuid: "19cd39281db54eaa8afee03ab266e879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19cd39281db54eaa8afee03ab266e879" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 46505 } } }
I20260812 00:39:15.148947 14244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:15.149072 14255 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:15.154959 14255 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:15.164742 14255 catalog_manager.cc:1347] Generated new cluster ID: 00d2dca79ddc49e3b0bab9dba017fdc6
I20260812 00:39:15.164888 14255 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:15.190629 14255 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:15.192016 14255 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:15.208380 14255 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879: Generated new TSK 0
I20260812 00:39:15.209385 14255 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:15.217633 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46505
--builtin_ntp_servers=127.7.244.84:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:39:15.586814 14271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:15.587143 14271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:15.587255 14271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:15.594916 14271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:15.595045 14271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:15.595194 14271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:39:15.604771 14271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:15.606220 14271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:15.608192 14271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:15.621572 14281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:15.621603 14282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:15.621665 14284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:15.622704 14271 server_base.cc:1034] running on GCE node
I20260812 00:39:15.623319 14271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:15.624274 14271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:15.625504 14271 hybrid_clock.cc:648] HybridClock initialized: now 1786495155625445 us; error 48 us; skew 500 ppm
I20260812 00:39:15.625849 14271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:15.628393 14271 webserver.cc:460] Webserver started at http://127.7.244.65:39909/ using document root <none> and password file <none>
I20260812 00:39:15.629168 14271 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:15.629348 14271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:15.629730 14271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:15.632875 14271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e"
format_stamp: "Formatted at 2026-08-12 00:39:15 on dist-test-slave-gjw7"
server_key: "c5fe3c276cb886e9f9cf1fb453b80d32"
server_key_iv: "cc9bceebc6f150c4c7492dc78e552bcf"
server_key_version: "encryptionkey@0"
I20260812 00:39:15.633796 14271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e"
format_stamp: "Formatted at 2026-08-12 00:39:15 on dist-test-slave-gjw7"
server_key: "c5fe3c276cb886e9f9cf1fb453b80d32"
server_key_iv: "cc9bceebc6f150c4c7492dc78e552bcf"
server_key_version: "encryptionkey@0"
I20260812 00:39:15.639904 14271 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 00:39:15.643954 14295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:15.645741 14271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 00:39:15.645939 14271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e"
format_stamp: "Formatted at 2026-08-12 00:39:15 on dist-test-slave-gjw7"
server_key: "c5fe3c276cb886e9f9cf1fb453b80d32"
server_key_iv: "cc9bceebc6f150c4c7492dc78e552bcf"
server_key_version: "encryptionkey@0"
I20260812 00:39:15.646234 14271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:15.681231 14271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:15.682265 14271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:15.682591 14271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:15.683672 14271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:15.685389 14271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:15.685506 14271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:15.685626 14271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:15.685702 14271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:15.718410 14271 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:42331
I20260812 00:39:15.718488 14518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:42331 every 8 connection(s)
I20260812 00:39:15.719877 14271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:39:15.722554 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 14271
I20260812 00:39:15.722723 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:39:15.723165 8145 external_mini_cluster.cc:1428] Setting key efd4160d4692acc3d3e5359e79922718
I20260812 00:39:15.726197 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46505
--builtin_ntp_servers=127.7.244.84:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:39:15.734148 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:15.722647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:58215 (local address 127.7.244.126:46505)
0812 00:39:15.723983 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:15.723987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:15.725021 (+ 1034us) server_negotiation.cc:407] Connection header received
0812 00:39:15.727071 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:15.727077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:15.727172 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:15.727294 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:15.728554 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:15.729613 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:15.731205 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:15.731750 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:15.731878 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:15.733275 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:15.733281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:15.733285 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:15.733343 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:15.733393 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:15.733409 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:15.733498 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:15.733706 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 00:39:15.733817 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":1080,"threads_started":1}
I20260812 00:39:15.735154 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:15.723382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46505 (local address 127.7.244.65:58215)
0812 00:39:15.724835 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:15.724899 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:15.726761 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:15.727617 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:15.727626 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:15.727727 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:15.728358 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:15.728377 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:15.729761 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:15.729765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:15.731045 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:15.731064 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:15.731966 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:15.731970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:15.732199 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:15.732919 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:15.732922 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:15.732968 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:15.733057 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:15.733500 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:15.733509 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:15.733515 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 00:39:15.733673 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 00:39:15.734009 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":587,"threads_started":1}
I20260812 00:39:15.736771 14519 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46505
I20260812 00:39:15.737076 14519 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:15.737923 14519 heartbeater.cc:511] Master 127.7.244.126:46505 requested a full tablet report, sending...
I20260812 00:39:15.740202 14158 ts_manager.cc:194] Registered new tserver with Master: f7ddffc376c74c289ae1b9bb3ebf0a6e (127.7.244.65:42331)
I20260812 00:39:15.741828 14158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.65:58215
W20260812 00:39:16.086095 14525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:16.086432 14525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:16.086575 14525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:16.094126 14525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:16.094249 14525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:16.094357 14525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:39:16.103989 14525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:16.105377 14525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:16.107270 14525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:16.121181 14540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:16.121219 14543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:16.121248 14537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:16.123585 14525 server_base.cc:1034] running on GCE node
I20260812 00:39:16.124264 14525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:16.125325 14525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:16.126559 14525 hybrid_clock.cc:648] HybridClock initialized: now 1786495156126495 us; error 59 us; skew 500 ppm
I20260812 00:39:16.126953 14525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:16.129567 14525 webserver.cc:460] Webserver started at http://127.7.244.66:38493/ using document root <none> and password file <none>
I20260812 00:39:16.130347 14525 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:16.130466 14525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:16.130824 14525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:16.134389 14525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "bd2553c1b54945159df52d6fab16504f"
format_stamp: "Formatted at 2026-08-12 00:39:16 on dist-test-slave-gjw7"
server_key: "d0ceb1274ce3ec87849d3cea67b91a5f"
server_key_iv: "1ec7565be61f53334538b90c29abd693"
server_key_version: "encryptionkey@0"
I20260812 00:39:16.135289 14525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "bd2553c1b54945159df52d6fab16504f"
format_stamp: "Formatted at 2026-08-12 00:39:16 on dist-test-slave-gjw7"
server_key: "d0ceb1274ce3ec87849d3cea67b91a5f"
server_key_iv: "1ec7565be61f53334538b90c29abd693"
server_key_version: "encryptionkey@0"
I20260812 00:39:16.140964 14525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260812 00:39:16.144732 14556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:16.146229 14525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:39:16.146438 14525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "bd2553c1b54945159df52d6fab16504f"
format_stamp: "Formatted at 2026-08-12 00:39:16 on dist-test-slave-gjw7"
server_key: "d0ceb1274ce3ec87849d3cea67b91a5f"
server_key_iv: "1ec7565be61f53334538b90c29abd693"
server_key_version: "encryptionkey@0"
I20260812 00:39:16.146669 14525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:16.177891 14525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:16.178843 14525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:16.179165 14525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:16.180275 14525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:16.182001 14525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:16.182101 14525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:16.182226 14525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:16.182292 14525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:16.214470 14525 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:41669
I20260812 00:39:16.214533 14809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:41669 every 8 connection(s)
I20260812 00:39:16.215833 14525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:39:16.219497 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 14525
I20260812 00:39:16.219729 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:39:16.220377 8145 external_mini_cluster.cc:1428] Setting key fae49b0d66c9c6adaeb716c04d933075
I20260812 00:39:16.224016 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46505
--builtin_ntp_servers=127.7.244.84:38479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:39:16.231829 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.218709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:52927 (local address 127.7.244.126:46505)
0812 00:39:16.218938 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.218944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.220797 (+ 1853us) server_negotiation.cc:407] Connection header received
0812 00:39:16.223115 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.223120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.223195 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.223280 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:16.225041 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.225877 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.228420 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.228886 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.229021 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.230899 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:16.230906 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:16.230909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:16.230932 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:16.230969 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:16.230983 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:16.231067 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.231459 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.231565 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 00:39:16.233505 14816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.219251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46505 (local address 127.7.244.66:52927)
0812 00:39:16.220569 (+ 1318us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:16.220628 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:16.222743 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:16.223631 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:16.223646 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:16.223791 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:16.224777 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.224820 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.226085 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.226092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.228150 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.228183 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.229161 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.229179 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.229562 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.230422 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:16.230426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:16.230493 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:16.230619 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:16.231099 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:16.231111 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:16.231121 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 00:39:16.231380 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 00:39:16.231889 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":381,"threads_started":1}
I20260812 00:39:16.235258 14813 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46505
I20260812 00:39:16.235622 14813 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:16.236615 14813 heartbeater.cc:511] Master 127.7.244.126:46505 requested a full tablet report, sending...
I20260812 00:39:16.238473 14158 ts_manager.cc:194] Registered new tserver with Master: bd2553c1b54945159df52d6fab16504f (127.7.244.66:41669)
I20260812 00:39:16.239310 14158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.66:52927
W20260812 00:39:16.588989 14817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:16.589370 14817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:16.589504 14817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:16.597283 14817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:16.597456 14817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:16.597647 14817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:39:16.607277 14817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:38479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:46505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:16.608759 14817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:16.610809 14817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:16.625517 14826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:16.625588 14830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:16.625554 14827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:16.626368 14817 server_base.cc:1034] running on GCE node
I20260812 00:39:16.627084 14817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:16.628115 14817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:16.629361 14817 hybrid_clock.cc:648] HybridClock initialized: now 1786495156629283 us; error 65 us; skew 500 ppm
I20260812 00:39:16.629714 14817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:16.632246 14817 webserver.cc:460] Webserver started at http://127.7.244.67:35023/ using document root <none> and password file <none>
I20260812 00:39:16.633075 14817 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:16.633186 14817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:16.633599 14817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:16.636713 14817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b"
format_stamp: "Formatted at 2026-08-12 00:39:16 on dist-test-slave-gjw7"
server_key: "7680b1c21c92bdbca849d34ac69ac23e"
server_key_iv: "c9c5181be056861ae3bfb108b26f2591"
server_key_version: "encryptionkey@0"
I20260812 00:39:16.637593 14817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b"
format_stamp: "Formatted at 2026-08-12 00:39:16 on dist-test-slave-gjw7"
server_key: "7680b1c21c92bdbca849d34ac69ac23e"
server_key_iv: "c9c5181be056861ae3bfb108b26f2591"
server_key_version: "encryptionkey@0"
I20260812 00:39:16.643121 14817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 00:39:16.647408 14848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:16.648979 14817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:39:16.649137 14817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b"
format_stamp: "Formatted at 2026-08-12 00:39:16 on dist-test-slave-gjw7"
server_key: "7680b1c21c92bdbca849d34ac69ac23e"
server_key_iv: "c9c5181be056861ae3bfb108b26f2591"
server_key_version: "encryptionkey@0"
I20260812 00:39:16.649441 14817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:16.677001 14817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:16.678001 14817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:16.678315 14817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:16.679411 14817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:16.681002 14817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:16.681098 14817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:16.681213 14817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:16.681279 14817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:16.719257 14817 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:33495
I20260812 00:39:16.719369 15084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:33495 every 8 connection(s)
I20260812 00:39:16.720640 14817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:39:16.730006 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 14817
I20260812 00:39:16.730187 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:39:16.730656 8145 external_mini_cluster.cc:1428] Setting key 5caa9be836b897968263f960ecb0e814
I20260812 00:39:16.734063 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.723512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:46319 (local address 127.7.244.126:46505)
0812 00:39:16.723734 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.723741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.725189 (+ 1448us) server_negotiation.cc:407] Connection header received
0812 00:39:16.727082 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.727087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.727167 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.727264 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:16.728456 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.729214 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.731098 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.731577 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.731715 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.733164 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:16.733171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:16.733176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:16.733198 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:16.733243 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:16.733260 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:16.733396 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.733655 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.733767 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:39:16.735169 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.724036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:46505 (local address 127.7.244.67:46319)
0812 00:39:16.725007 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:16.725046 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:16.726789 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:16.727489 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:16.727498 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:16.727598 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:16.728235 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.728254 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.729411 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.729415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.730919 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.730935 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.731806 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.731812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.732128 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.732815 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:16.732818 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:16.732869 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:16.732963 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:16.733419 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:16.733429 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:16.733436 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 00:39:16.733635 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 00:39:16.734041 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":297,"threads_started":1}
I20260812 00:39:16.736420 15087 heartbeater.cc:347] Connected to a master server at 127.7.244.126:46505
I20260812 00:39:16.736682 15087 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:16.737450 15087 heartbeater.cc:511] Master 127.7.244.126:46505 requested a full tablet report, sending...
I20260812 00:39:16.738797 14158 ts_manager.cc:194] Registered new tserver with Master: a9b6ed86dd1247b2b8db8cc3cfaaa06b (127.7.244.67:33495)
I20260812 00:39:16.739635 14158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.67:46319
I20260812 00:39:16.745154 14519 heartbeater.cc:503] Master 127.7.244.126:46505 was elected leader, sending a full tablet report...
I20260812 00:39:16.746825 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:39:16.757891 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.751211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60012 (local address 127.7.244.126:46505)
0812 00:39:16.751444 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.751451 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.751919 (+ 468us) server_negotiation.cc:407] Connection header received
0812 00:39:16.752061 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.752064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.752136 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.752223 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:16.752855 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.753714 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.755285 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.755746 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.755868 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.757058 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:16.757063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:16.757066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:16.757088 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:16.757126 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:16.757138 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:16.757263 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.757494 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.757560 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 00:39:16.762202 14158 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:39:16.769413 14158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:39:16.795537 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.783785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.0.0.1:35486)
0812 00:39:16.785735 (+ 1950us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:16.785757 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:16.786766 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:16.787444 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:16.787449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:16.787475 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:16.787960 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.787980 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.789808 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.789814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.792917 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.792926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.794452 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.794459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.794655 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.794713 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 00:39:16.794870 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 00:39:16.795097 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1809,"mutex_wait_us":239,"thread_start_us":1940,"threads_started":1}
I20260812 00:39:16.795809 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.784484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35486 (local address 127.7.244.65:42331)
0812 00:39:16.785137 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.785147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.785882 (+ 735us) server_negotiation.cc:407] Connection header received
0812 00:39:16.786952 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.786961 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.787221 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.788170 (+ 949us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:16.788428 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.789676 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.793182 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.794231 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.794367 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.794686 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.795219 (+ 533us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.795425 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":346,"threads_started":1}
I20260812 00:39:16.795607 15110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.783388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:33495 (local address 127.0.0.1:45952)
0812 00:39:16.784315 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:16.784367 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:16.784550 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:16.787249 (+ 2699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:16.787257 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:16.787295 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:16.787817 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.787832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.789912 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.789917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.792892 (+ 2975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.792914 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.794857 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.794863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.795058 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.795089 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 00:39:16.795175 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 00:39:16.795308 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":249,"thread_start_us":371,"threads_started":1}
I20260812 00:39:16.796362 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.786232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59332 (local address 127.7.244.66:41669)
0812 00:39:16.786779 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.786786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.787566 (+ 780us) server_negotiation.cc:407] Connection header received
0812 00:39:16.787687 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.787695 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.787931 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.788109 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:16.789017 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.790387 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.793162 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.794671 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.794812 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.795046 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.795825 (+ 779us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.796028 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":299,"threads_started":1}
I20260812 00:39:16.796839 15118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.786133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:41669 (local address 127.0.0.1:59332)
0812 00:39:16.787258 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:16.787281 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:16.787415 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:16.788372 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:16.788376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:16.788402 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:16.788800 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.788813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.790541 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.790546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.792872 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.792891 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.795201 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.795207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.795391 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.795443 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 00:39:16.795545 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 00:39:16.795687 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"mutex_wait_us":114,"thread_start_us":882,"threads_started":1}
I20260812 00:39:16.797591 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.783719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45952 (local address 127.7.244.67:33495)
0812 00:39:16.786488 (+ 2769us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.786498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.786537 (+ 39us) server_negotiation.cc:407] Connection header received
0812 00:39:16.786662 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.786671 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.786975 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.787250 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:16.788101 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.789776 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.793177 (+ 3401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.794671 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.794813 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.795081 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.797095 (+ 2014us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.797275 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2490,"thread_start_us":2346,"threads_started":1}
I20260812 00:39:16.800295 14654 tablet_service.cc:1467] Processing CreateTablet for tablet ed153cf14c044d00ab9fafb6ba28eee3 (DEFAULT_TABLE table=test-table [id=5310ea11a38b49c0b46891d5035c3e7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:16.800330 14378 tablet_service.cc:1467] Processing CreateTablet for tablet ed153cf14c044d00ab9fafb6ba28eee3 (DEFAULT_TABLE table=test-table [id=5310ea11a38b49c0b46891d5035c3e7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:16.801676 14654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed153cf14c044d00ab9fafb6ba28eee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:16.801183 14953 tablet_service.cc:1467] Processing CreateTablet for tablet ed153cf14c044d00ab9fafb6ba28eee3 (DEFAULT_TABLE table=test-table [id=5310ea11a38b49c0b46891d5035c3e7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:16.801868 14378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed153cf14c044d00ab9fafb6ba28eee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:16.802703 14953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed153cf14c044d00ab9fafb6ba28eee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:16.813393 15123 tablet_bootstrap.cc:492] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Bootstrap starting.
I20260812 00:39:16.816851 15124 tablet_bootstrap.cc:492] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: Bootstrap starting.
I20260812 00:39:16.817322 15123 tablet_bootstrap.cc:654] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:16.818871 15123 log.cc:824] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:16.820499 15125 tablet_bootstrap.cc:492] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Bootstrap starting.
I20260812 00:39:16.820806 15124 tablet_bootstrap.cc:654] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:16.822201 15123 tablet_bootstrap.cc:492] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: No bootstrap required, opened a new log
I20260812 00:39:16.822165 15124 log.cc:824] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:16.822489 15123 ts_tablet_manager.cc:1397] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260812 00:39:16.824306 15125 tablet_bootstrap.cc:654] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:16.825567 15124 tablet_bootstrap.cc:492] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: No bootstrap required, opened a new log
I20260812 00:39:16.825636 15125 log.cc:824] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:16.825884 15124 ts_tablet_manager.cc:1397] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 00:39:16.828998 15125 tablet_bootstrap.cc:492] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: No bootstrap required, opened a new log
I20260812 00:39:16.829324 15125 ts_tablet_manager.cc:1397] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 00:39:16.831081 15123 raft_consensus.cc:348] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.831475 15123 raft_consensus.cc:374] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:16.831573 15123 raft_consensus.cc:729] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd2553c1b54945159df52d6fab16504f, State: Initialized, Role: FOLLOWER
I20260812 00:39:16.832190 15123 consensus_queue.cc:260] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.835253 14813 heartbeater.cc:503] Master 127.7.244.126:46505 was elected leader, sending a full tablet report...
I20260812 00:39:16.836284 15124 raft_consensus.cc:348] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.836807 15124 raft_consensus.cc:374] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:16.836911 15124 raft_consensus.cc:729] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ddffc376c74c289ae1b9bb3ebf0a6e, State: Initialized, Role: FOLLOWER
I20260812 00:39:16.837630 15124 consensus_queue.cc:260] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.839591 15124 ts_tablet_manager.cc:1428] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260812 00:39:16.839852 15125 raft_consensus.cc:348] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.840317 15125 raft_consensus.cc:374] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:16.840418 15125 raft_consensus.cc:729] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9b6ed86dd1247b2b8db8cc3cfaaa06b, State: Initialized, Role: FOLLOWER
I20260812 00:39:16.840909 15123 ts_tablet_manager.cc:1428] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Time spent starting tablet: real 0.018s user 0.006s sys 0.007s
I20260812 00:39:16.841008 15125 consensus_queue.cc:260] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.842386 15087 heartbeater.cc:503] Master 127.7.244.126:46505 was elected leader, sending a full tablet report...
I20260812 00:39:16.842875 15125 ts_tablet_manager.cc:1428] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
I20260812 00:39:16.919938 15141 raft_consensus.cc:482] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:39:16.920272 15141 raft_consensus.cc:504] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.922166 15141 leader_election.cc:288] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7ddffc376c74c289ae1b9bb3ebf0a6e (127.7.244.65:42331), bd2553c1b54945159df52d6fab16504f (127.7.244.66:41669)
I20260812 00:39:16.927973 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.922669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.7.244.67:36903)
0812 00:39:16.922989 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:16.923005 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:16.923166 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:16.923560 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:16.923565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:16.923595 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:16.923763 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.923771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.924741 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.924745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.926144 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.926157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.927252 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.927259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.927424 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.927449 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 00:39:16.927531 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 00:39:16.927666 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214}
I20260812 00:39:16.927970 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.922602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:36903 (local address 127.7.244.65:42331)
0812 00:39:16.922839 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.922845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.923099 (+ 254us) server_negotiation.cc:407] Connection header received
0812 00:39:16.923324 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.923329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.923407 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.923482 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:16.923924 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.924603 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.926309 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.927095 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.927252 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.927330 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.927591 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.927724 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 00:39:16.928445 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.922712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:33301 (local address 127.7.244.66:41669)
0812 00:39:16.922933 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.922938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.923624 (+ 686us) server_negotiation.cc:407] Connection header received
0812 00:39:16.923822 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.923827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.923902 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.923980 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:16.924479 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.925159 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.926845 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.927573 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.927723 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.927790 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.928053 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.928196 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 00:39:16.928769 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.922704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:41669 (local address 127.7.244.67:33301)
0812 00:39:16.923511 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:16.923525 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:16.923687 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:16.924015 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:16.924020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:16.924042 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:16.924333 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.924342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.925347 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.925356 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.926668 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:16.926681 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.927735 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.927741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:16.927897 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.927921 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 00:39:16.927997 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 00:39:16.928446 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"spinlock_wait_cycles":2944,"thread_start_us":450,"threads_started":1}
I20260812 00:39:16.929142 14425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed153cf14c044d00ab9fafb6ba28eee3" candidate_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" is_pre_election: true
I20260812 00:39:16.929808 14425 raft_consensus.cc:2454] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b6ed86dd1247b2b8db8cc3cfaaa06b in term 0.
I20260812 00:39:16.929845 14705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed153cf14c044d00ab9fafb6ba28eee3" candidate_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd2553c1b54945159df52d6fab16504f" is_pre_election: true
I20260812 00:39:16.930384 14705 raft_consensus.cc:2454] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9b6ed86dd1247b2b8db8cc3cfaaa06b in term 0.
I20260812 00:39:16.930665 14852 leader_election.cc:302] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9b6ed86dd1247b2b8db8cc3cfaaa06b, f7ddffc376c74c289ae1b9bb3ebf0a6e; no voters:
I20260812 00:39:16.931109 15141 raft_consensus.cc:2781] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:39:16.931244 15141 raft_consensus.cc:482] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:39:16.931339 15141 raft_consensus.cc:3037] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:16.933681 15141 raft_consensus.cc:504] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.934353 15141 leader_election.cc:288] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [CANDIDATE]: Term 1 election: Requested vote from peers f7ddffc376c74c289ae1b9bb3ebf0a6e (127.7.244.65:42331), bd2553c1b54945159df52d6fab16504f (127.7.244.66:41669)
I20260812 00:39:16.934974 14705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed153cf14c044d00ab9fafb6ba28eee3" candidate_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd2553c1b54945159df52d6fab16504f"
I20260812 00:39:16.934986 14425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed153cf14c044d00ab9fafb6ba28eee3" candidate_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e"
I20260812 00:39:16.935223 14425 raft_consensus.cc:3037] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:16.935223 14705 raft_consensus.cc:3037] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:16.938683 14425 raft_consensus.cc:2454] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b6ed86dd1247b2b8db8cc3cfaaa06b in term 1.
I20260812 00:39:16.938683 14705 raft_consensus.cc:2454] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9b6ed86dd1247b2b8db8cc3cfaaa06b in term 1.
I20260812 00:39:16.939265 14852 leader_election.cc:302] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a9b6ed86dd1247b2b8db8cc3cfaaa06b, bd2553c1b54945159df52d6fab16504f, f7ddffc376c74c289ae1b9bb3ebf0a6e; no voters:
I20260812 00:39:16.939658 15141 raft_consensus.cc:2781] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:16.940126 15141 raft_consensus.cc:686] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 1 LEADER]: Becoming Leader. State: Replica: a9b6ed86dd1247b2b8db8cc3cfaaa06b, State: Running, Role: LEADER
I20260812 00:39:16.940645 15141 consensus_queue.cc:237] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } }
I20260812 00:39:16.945657 14158 catalog_manager.cc:5495] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b reported cstate change: term changed from 0 to 1, leader changed from <none> to a9b6ed86dd1247b2b8db8cc3cfaaa06b (127.7.244.67). New cstate: current_term: 1 leader_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9b6ed86dd1247b2b8db8cc3cfaaa06b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 33495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 } health_report { overall_health: UNKNOWN } } }
I20260812 00:39:16.963833 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:16.957784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45962 (local address 127.7.244.67:33495)
0812 00:39:16.958011 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:16.958017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:16.958112 (+ 95us) server_negotiation.cc:407] Connection header received
0812 00:39:16.958360 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:16.958364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:16.958456 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:16.958597 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:39:16.959024 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.959765 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.961253 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:16.961738 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:16.961867 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:16.962543 (+ 676us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:39:16.963083 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:39:16.963266 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:16.963484 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 00:39:16.963533 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 00:39:16.968025 14815 tablet.cc:2369] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:39:16.972277 14521 tablet.cc:2369] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:39:16.972580 15090 tablet.cc:2369] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:16.972991 14705 raft_consensus.cc:1261] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 1 FOLLOWER]: Refusing update from remote peer a9b6ed86dd1247b2b8db8cc3cfaaa06b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:16.973002 14425 raft_consensus.cc:1261] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 1 FOLLOWER]: Refusing update from remote peer a9b6ed86dd1247b2b8db8cc3cfaaa06b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:16.974151 15145 consensus_queue.cc:1035] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ddffc376c74c289ae1b9bb3ebf0a6e" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 42331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:16.974536 15141 consensus_queue.cc:1035] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd2553c1b54945159df52d6fab16504f" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 41669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:16.989224 15158 mvcc.cc:205] Tried to move back new op lower bound from 7317484162949619712 to 7317484162833117184. Current Snapshot: MvccSnapshot[applied={T|T < 7317484162949619712}]
I20260812 00:39:17.002115 14158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:17.002375 14158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:17.006105 14158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19cd39281db54eaa8afee03ab266e879: Sending DeleteTablet for 3 replicas of tablet ed153cf14c044d00ab9fafb6ba28eee3
I20260812 00:39:17.007540 14378 tablet_service.cc:1514] Processing DeleteTablet for tablet ed153cf14c044d00ab9fafb6ba28eee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:16 UTC) from {username='slave'} at 127.0.0.1:35486
I20260812 00:39:17.007659 14953 tablet_service.cc:1514] Processing DeleteTablet for tablet ed153cf14c044d00ab9fafb6ba28eee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:16 UTC) from {username='slave'} at 127.0.0.1:45952
I20260812 00:39:17.007931 14654 tablet_service.cc:1514] Processing DeleteTablet for tablet ed153cf14c044d00ab9fafb6ba28eee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:16 UTC) from {username='slave'} at 127.0.0.1:59332
I20260812 00:39:17.013967 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14271
I20260812 00:39:17.022707 15179 tablet_replica.cc:330] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: stopping tablet replica
I20260812 00:39:17.023315 15179 raft_consensus.cc:2229] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:17.023973 15179 raft_consensus.cc:2258] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:17.028600 15174 tablet_replica.cc:330] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: stopping tablet replica
I20260812 00:39:17.029160 15174 raft_consensus.cc:2229] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:17.029320 15177 tablet_replica.cc:330] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: stopping tablet replica
I20260812 00:39:17.029697 15174 raft_consensus.cc:2258] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:17.029783 15177 raft_consensus.cc:2229] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:39:17.030841 15177 raft_consensus.cc:2258] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:17.031473 15179 ts_tablet_manager.cc:1905] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:17.033587 15177 ts_tablet_manager.cc:1905] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:17.034049 15117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.015667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35494 (local address 127.7.244.65:42331)
0812 00:39:17.015884 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:17.015891 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:17.017341 (+ 1450us) server_negotiation.cc:407] Connection header received
0812 00:39:17.017452 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:17.017459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:17.017584 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:17.017692 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:17.018767 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:17.020040 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:17.025825 (+ 5785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:17.026655 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:17.026786 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:17.031807 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:17.031822 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:17.031841 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:17.032731 (+ 890us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:17.032819 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:17.032856 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:17.033028 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:17.033379 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 00:39:17.033531 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:39:17.034826 14506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:17.041599 15177 ts_tablet_manager.cc:1918] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:17.041849 15177 log.cc:1196] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/ed153cf14c044d00ab9fafb6ba28eee3
I20260812 00:39:17.042078 15179 ts_tablet_manager.cc:1918] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:17.042269 15179 log.cc:1196] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/ed153cf14c044d00ab9fafb6ba28eee3
I20260812 00:39:17.042505 15177 ts_tablet_manager.cc:1939] T ed153cf14c044d00ab9fafb6ba28eee3 P a9b6ed86dd1247b2b8db8cc3cfaaa06b: Deleting consensus metadata
I20260812 00:39:17.042812 15179 ts_tablet_manager.cc:1939] T ed153cf14c044d00ab9fafb6ba28eee3 P bd2553c1b54945159df52d6fab16504f: Deleting consensus metadata
I20260812 00:39:17.044718 14136 catalog_manager.cc:4841] TS a9b6ed86dd1247b2b8db8cc3cfaaa06b (127.7.244.67:33495): tablet ed153cf14c044d00ab9fafb6ba28eee3 (table test-table [id=5310ea11a38b49c0b46891d5035c3e7e]) successfully deleted
I20260812 00:39:17.045113 14136 catalog_manager.cc:4841] TS bd2553c1b54945159df52d6fab16504f (127.7.244.66:41669): tablet ed153cf14c044d00ab9fafb6ba28eee3 (table test-table [id=5310ea11a38b49c0b46891d5035c3e7e]) successfully deleted
I20260812 00:39:17.033663 15174 ts_tablet_manager.cc:1905] T ed153cf14c044d00ab9fafb6ba28eee3 P f7ddffc376c74c289ae1b9bb3ebf0a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:17.144390 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 14271
W20260812 00:39:17.177870 14134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.244.65:42331 (error 108)
I20260812 00:39:17.178089 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14525
I20260812 00:39:17.180095 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.179560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.0.0.1:35504)
0812 00:39:17.179755 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:17.179902 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:42331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 00:39:17.180401 14134 catalog_manager.cc:4584] TS f7ddffc376c74c289ae1b9bb3ebf0a6e (127.7.244.65:42331): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed153cf14c044d00ab9fafb6ba28eee3: Network error: Client connection negotiation failed: client connection to 127.7.244.65:42331: connect: Connection refused (error 111)
I20260812 00:39:17.187013 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.179581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.7.244.66:41669)
0812 00:39:17.179755 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:17.179760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:17.179780 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:39:17.179854 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:17.179859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:17.179970 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:17.180087 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:17.180625 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:17.181497 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:17.183574 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:17.184015 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:17.184141 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:17.185571 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:17.185582 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:17.185599 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:17.186223 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:17.186277 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:17.186302 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:17.186400 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:17.186570 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 00:39:17.186684 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 00:39:17.187747 14794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:17.234334 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.233706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.0.0.1:35506)
0812 00:39:17.233998 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:17.234120 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:42331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 00:39:17.235693 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.235212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.0.0.1:35514)
0812 00:39:17.235402 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:17.235504 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:42331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 00:39:17.278858 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 14525
I20260812 00:39:17.296869 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.296180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.0.0.1:35526)
0812 00:39:17.296505 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:17.296644 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:42331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260812 00:39:17.298338 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.297817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.0.0.1:35536)
0812 00:39:17.298024 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:17.298137 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:42331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 00:39:17.301538 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14817
I20260812 00:39:17.309150 15114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.302472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.7.244.67:33495)
0812 00:39:17.302707 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:17.302713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:17.302735 (+ 22us) server_negotiation.cc:407] Connection header received
0812 00:39:17.302908 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:17.302929 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:17.303034 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:17.303123 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:17.303589 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:17.304403 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:17.305810 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:17.306283 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:17.306405 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:17.307636 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:17.307645 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:17.307666 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:17.308337 (+ 671us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:17.308393 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:17.308416 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:17.308519 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:17.308675 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 00:39:17.308788 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 00:39:17.309957 15080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:17.379297 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.378752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.0.0.1:35546)
0812 00:39:17.378990 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:17.379131 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:42331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260812 00:39:17.380579 15115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:17.380135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:42331 (local address 127.0.0.1:35548)
0812 00:39:17.380363 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:17.380449 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:42331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260812 00:39:17.434358 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 14817
I20260812 00:39:17.462013 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 14104
I20260812 00:39:17.463274 14218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:17.594535 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 14104
2026-08-12T00:39:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3055 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:17 dist-test-slave-gjw7 krb5kdc[15198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:17 dist-test-slave-gjw7 krb5kdc[15198](info): set up 2 sockets
Aug 12 00:39:17 dist-test-slave-gjw7 krb5kdc[15198](info): commencing operation
krb5kdc: starting...
W20260812 00:39:18.952127 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:19 dist-test-slave-gjw7 krb5kdc[15198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:39:19.276844 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33029
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:40565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:33029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:19.642482 15223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:19.642809 15223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:19.642920 15223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:19.650520 15223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:19.650655 15223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:19.650702 15223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:19.650767 15223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:19.650812 15223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:19.661164 15223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:40565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:33029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:33029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:19.662591 15223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:19.663987 15223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:19.676926 15236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:19.676980 15244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:19.676947 15238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:19.677979 15223 server_base.cc:1034] running on GCE node
I20260812 00:39:19.678936 15223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:19.680691 15223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:19.681941 15223 hybrid_clock.cc:648] HybridClock initialized: now 1786495159681872 us; error 70 us; skew 500 ppm
I20260812 00:39:19.682360 15223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:19 dist-test-slave-gjw7 krb5kdc[15198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495159, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:19.689759 15223 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:39:19.691541 15223 webserver.cc:460] Webserver started at http://127.7.244.126:46699/ using document root <none> and password file <none>
I20260812 00:39:19.692342 15223 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:19.692487 15223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:19.692834 15223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:19.696281 15223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "04b3caede9204fbd88a381f2bae111a7"
format_stamp: "Formatted at 2026-08-12 00:39:19 on dist-test-slave-gjw7"
server_key: "0bd28bc975e861ef68edeff9a28e8ef4"
server_key_iv: "87db5d949b1b2096bbb5fda507c0c10a"
server_key_version: "encryptionkey@0"
I20260812 00:39:19.697193 15223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "04b3caede9204fbd88a381f2bae111a7"
format_stamp: "Formatted at 2026-08-12 00:39:19 on dist-test-slave-gjw7"
server_key: "0bd28bc975e861ef68edeff9a28e8ef4"
server_key_iv: "87db5d949b1b2096bbb5fda507c0c10a"
server_key_version: "encryptionkey@0"
I20260812 00:39:19.703234 15223 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 00:39:19.707418 15260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:19.709003 15223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:39:19.709201 15223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "04b3caede9204fbd88a381f2bae111a7"
format_stamp: "Formatted at 2026-08-12 00:39:19 on dist-test-slave-gjw7"
server_key: "0bd28bc975e861ef68edeff9a28e8ef4"
server_key_iv: "87db5d949b1b2096bbb5fda507c0c10a"
server_key_version: "encryptionkey@0"
I20260812 00:39:19.709512 15223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:19.726249 15223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:19.731083 15223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:19.731401 15223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:19.750689 15223 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:33029
I20260812 00:39:19.750741 15373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:33029 every 8 connection(s)
I20260812 00:39:19.752374 15223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:19.756619 15374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:19.759528 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 15223
I20260812 00:39:19.759699 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:19.760157 8145 external_mini_cluster.cc:1428] Setting key 21f8a1e35fc24bc542c7c5d388a4a4de
I20260812 00:39:19.770614 15374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7: Bootstrap starting.
I20260812 00:39:19.774595 15374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:19.775740 15374 log.cc:824] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:39:19 dist-test-slave-gjw7 krb5kdc[15198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:19.779570 15374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7: No bootstrap required, opened a new log
I20260812 00:39:19.788947 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:19.762361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39504 (local address 127.7.244.126:33029)
0812 00:39:19.763276 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:19.763295 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:19.763354 (+ 59us) server_negotiation.cc:407] Connection header received
0812 00:39:19.765874 (+ 2520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:19.765910 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:19.766528 (+ 618us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:19.767084 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:19.768649 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:19.770787 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:19.772268 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:19.773399 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:19.773602 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:19.778783 (+ 5181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:19.778797 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:19.778820 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:19.778872 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:19.783344 (+ 4472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:19.784295 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:19.784305 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:19.784316 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:19.784573 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:19.785038 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:19.785043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:19.785047 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:19.785975 (+ 928us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:19.786326 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:19.786824 (+ 498us) server_negotiation.cc:299] Negotiation successful
0812 00:39:19.787374 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":382,"threads_started":1}
I20260812 00:39:19.790841 15374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b3caede9204fbd88a381f2bae111a7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33029 } }
I20260812 00:39:19.791430 15374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:19.791587 15374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04b3caede9204fbd88a381f2bae111a7, State: Initialized, Role: FOLLOWER
I20260812 00:39:19.792250 15374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b3caede9204fbd88a381f2bae111a7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33029 } }
I20260812 00:39:19.792574 15374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:19.792742 15374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:19.792932 15374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:19.795454 15374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b3caede9204fbd88a381f2bae111a7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33029 } }
I20260812 00:39:19.796088 15374 leader_election.cc:302] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04b3caede9204fbd88a381f2bae111a7; no voters:
I20260812 00:39:19.796748 15374 leader_election.cc:288] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:19.796891 15389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:19.797631 15389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [term 1 LEADER]: Becoming Leader. State: Replica: 04b3caede9204fbd88a381f2bae111a7, State: Running, Role: LEADER
I20260812 00:39:19.798249 15389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b3caede9204fbd88a381f2bae111a7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33029 } }
I20260812 00:39:19.799111 15374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:19.801748 15393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04b3caede9204fbd88a381f2bae111a7. Latest consensus state: current_term: 1 leader_uuid: "04b3caede9204fbd88a381f2bae111a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b3caede9204fbd88a381f2bae111a7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33029 } } }
I20260812 00:39:19.802264 15393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:19.804031 15390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04b3caede9204fbd88a381f2bae111a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04b3caede9204fbd88a381f2bae111a7" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 33029 } } }
I20260812 00:39:19.804432 15390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:19.805047 15410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:19.810945 15410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:19.820914 15410 catalog_manager.cc:1347] Generated new cluster ID: d0b81dd560f74e7a8094d8a5d2dbc654
I20260812 00:39:19.821053 15410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:19.832844 15410 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:19.834362 15410 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:19.842139 15410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7: Generated new TSK 0
I20260812 00:39:19.843137 15410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.244.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
Entry for principal kudu/127.7.244.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab.
I20260812 00:39:19.916651 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33029
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:40565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:20.285531 15434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:20.285846 15434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:20.285989 15434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:20.293963 15434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:20.294131 15434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:20.294304 15434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:39:20.305075 15434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:40565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.65.keytab
--principal=kudu/127.7.244.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:20.306546 15434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:20.307991 15434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:20.322366 15450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:20.322422 15446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:20.322304 15447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:20.323418 15434 server_base.cc:1034] running on GCE node
I20260812 00:39:20.324110 15434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:20.325212 15434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:20.326576 15434 hybrid_clock.cc:648] HybridClock initialized: now 1786495160326490 us; error 75 us; skew 500 ppm
I20260812 00:39:20.326968 15434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:20 dist-test-slave-gjw7 krb5kdc[15198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:20.330612 15434 init.cc:377] Logged in from keytab as kudu/127.7.244.65@KRBTEST.COM (short username kudu)
I20260812 00:39:20.332440 15434 webserver.cc:460] Webserver started at http://127.7.244.65:41395/ using document root <none> and password file <none>
I20260812 00:39:20.333209 15434 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:20.333389 15434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:20.333765 15434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:20.337168 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "5b1d4dd44d3e401a9532b89ca809dab3"
format_stamp: "Formatted at 2026-08-12 00:39:20 on dist-test-slave-gjw7"
server_key: "d9fe6ae25ff4610999f6845f12c24ba3"
server_key_iv: "208d981871d1b86e78bb5d29f5c82e54"
server_key_version: "encryptionkey@0"
I20260812 00:39:20.338308 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "5b1d4dd44d3e401a9532b89ca809dab3"
format_stamp: "Formatted at 2026-08-12 00:39:20 on dist-test-slave-gjw7"
server_key: "d9fe6ae25ff4610999f6845f12c24ba3"
server_key_iv: "208d981871d1b86e78bb5d29f5c82e54"
server_key_version: "encryptionkey@0"
I20260812 00:39:20.344502 15434 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:39:20.349190 15464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:20.350947 15434 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260812 00:39:20.351142 15434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "5b1d4dd44d3e401a9532b89ca809dab3"
format_stamp: "Formatted at 2026-08-12 00:39:20 on dist-test-slave-gjw7"
server_key: "d9fe6ae25ff4610999f6845f12c24ba3"
server_key_iv: "208d981871d1b86e78bb5d29f5c82e54"
server_key_version: "encryptionkey@0"
I20260812 00:39:20.351370 15434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:20.373867 15434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:20.378883 15434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:20.379155 15434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:20.380193 15434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:20.381719 15434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:20.381824 15434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:20.381918 15434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:20.381963 15434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:20.415635 15434 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:44015
I20260812 00:39:20.415688 15712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:44015 every 8 connection(s)
I20260812 00:39:20.417073 15434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:39:20.419428 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 15434
I20260812 00:39:20.419636 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:39:20.420202 8145 external_mini_cluster.cc:1428] Setting key f3d440c875de4b23b3dcae7538e86189
Aug 12 00:39:20 dist-test-slave-gjw7 krb5kdc[15198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.65@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:20.439224 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:20.419779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:56383 (local address 127.7.244.126:33029)
0812 00:39:20.420231 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:20.420235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:20.421637 (+ 1402us) server_negotiation.cc:407] Connection header received
0812 00:39:20.423815 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:20.423821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:20.423923 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:20.424021 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:20.425821 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:20.426566 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:20.428197 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:20.428607 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:20.428725 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:20.432575 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:20.432580 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:20.432584 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:20.432621 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:20.435721 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:20.436616 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:20.436621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:20.436623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:20.436758 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:20.437232 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:20.437236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:20.437238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:20.437561 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:20.437717 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:20.438736 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0812 00:39:20.438944 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":219,"threads_started":1}
I20260812 00:39:20.440753 15724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:20.420487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33029 (local address 127.7.244.65:56383)
0812 00:39:20.421441 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:20.421508 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:20.423508 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:20.424303 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:20.424313 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:20.424917 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:20.425625 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:20.425651 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:20.426739 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:20.426743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:20.428031 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:20.428046 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:20.428855 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:20.428864 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:20.429106 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:20.429818 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:20.429843 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:20.432289 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:20.435915 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:20.435935 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:20.435952 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:20.436466 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:20.436877 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:20.436882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:20.436886 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:20.437101 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:20.437724 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:20.437733 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:20.438255 (+ 522us) client_negotiation.cc:769] Sending connection context
0812 00:39:20.438635 (+ 380us) client_negotiation.cc:240] Negotiation successful
0812 00:39:20.439227 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":355,"threads_started":1}
I20260812 00:39:20.442441 15718 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33029
I20260812 00:39:20.442780 15718 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:20.443549 15718 heartbeater.cc:511] Master 127.7.244.126:33029 requested a full tablet report, sending...
I20260812 00:39:20.445825 15291 ts_manager.cc:194] Registered new tserver with Master: 5b1d4dd44d3e401a9532b89ca809dab3 (127.7.244.65:44015)
I20260812 00:39:20.447407 15291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.65@KRBTEST.COM'} at 127.7.244.65:56383
WARNING: no policy specified for kudu/127.7.244.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
Entry for principal kudu/127.7.244.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab.
I20260812 00:39:20.488977 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33029
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:40565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:20.844038 15728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:20.844357 15728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:20.844486 15728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:20.852264 15728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:20.852439 15728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:20.852607 15728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:39:20.863457 15728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:40565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.66.keytab
--principal=kudu/127.7.244.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:20.864884 15728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:20.866376 15728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:20.880631 15742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:20.880770 15740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:20.880736 15751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:20.881719 15728 server_base.cc:1034] running on GCE node
I20260812 00:39:20.882437 15728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:20.883529 15728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:20.884754 15728 hybrid_clock.cc:648] HybridClock initialized: now 1786495160884697 us; error 48 us; skew 500 ppm
I20260812 00:39:20.885104 15728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:20 dist-test-slave-gjw7 krb5kdc[15198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:20.888490 15728 init.cc:377] Logged in from keytab as kudu/127.7.244.66@KRBTEST.COM (short username kudu)
I20260812 00:39:20.890149 15728 webserver.cc:460] Webserver started at http://127.7.244.66:37989/ using document root <none> and password file <none>
I20260812 00:39:20.890872 15728 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:20.891002 15728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:20.891342 15728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:20.894661 15728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "b5bbb0070f3f41e686d276e92b9a218b"
format_stamp: "Formatted at 2026-08-12 00:39:20 on dist-test-slave-gjw7"
server_key: "2a52b57b7800cb92dc2a4924fa6dec21"
server_key_iv: "01dac4235b19b14082e0d5f5ef256362"
server_key_version: "encryptionkey@0"
I20260812 00:39:20.895624 15728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "b5bbb0070f3f41e686d276e92b9a218b"
format_stamp: "Formatted at 2026-08-12 00:39:20 on dist-test-slave-gjw7"
server_key: "2a52b57b7800cb92dc2a4924fa6dec21"
server_key_iv: "01dac4235b19b14082e0d5f5ef256362"
server_key_version: "encryptionkey@0"
I20260812 00:39:20.901976 15728 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260812 00:39:20.906327 15771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:20.907992 15728 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 00:39:20.908179 15728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "b5bbb0070f3f41e686d276e92b9a218b"
format_stamp: "Formatted at 2026-08-12 00:39:20 on dist-test-slave-gjw7"
server_key: "2a52b57b7800cb92dc2a4924fa6dec21"
server_key_iv: "01dac4235b19b14082e0d5f5ef256362"
server_key_version: "encryptionkey@0"
I20260812 00:39:20.908388 15728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:20.923709 15728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:20.928418 15728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:20.928696 15728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:20.929802 15728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:20.931275 15728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:20.931352 15728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:20.931440 15728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:20.931488 15728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:20.963865 15728 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:43289
I20260812 00:39:20.963900 16011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:43289 every 8 connection(s)
I20260812 00:39:20.965363 15728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:39:20.971257 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 15728
I20260812 00:39:20.971463 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:39:20.972069 8145 external_mini_cluster.cc:1428] Setting key 00789f51522ae1b8f600630ed047c60b
Aug 12 00:39:20 dist-test-slave-gjw7 krb5kdc[15198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495160, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.66@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:20.989002 16015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:20.968419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:51043 (local address 127.7.244.126:33029)
0812 00:39:20.968927 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:20.968932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:20.970033 (+ 1101us) server_negotiation.cc:407] Connection header received
0812 00:39:20.972559 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:20.972565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:20.972656 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:20.972784 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:20.974906 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:20.975688 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:20.978136 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:20.978623 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:20.978761 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:20.982707 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:20.982713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:20.982716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:20.982757 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:20.985446 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:20.986339 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:20.986344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:20.986347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:20.986551 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:20.987047 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:20.987053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:20.987056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:20.987367 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:20.987522 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:20.988515 (+ 993us) server_negotiation.cc:299] Negotiation successful
0812 00:39:20.988719 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":305,"threads_started":1}
I20260812 00:39:20.990265 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:20.968924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33029 (local address 127.7.244.66:51043)
0812 00:39:20.969832 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:20.969884 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:20.972160 (+ 2276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:20.973067 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:20.973082 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:20.973802 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:20.974655 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:20.974688 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:20.975886 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:20.975891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:20.977922 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:20.977942 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:20.978930 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:20.978941 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:20.979186 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:20.979930 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:20.979957 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:20.982379 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:20.985635 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:20.985646 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:20.985662 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:20.986159 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:20.986676 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:20.986681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:20.986684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:20.986900 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:20.987517 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:20.987526 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:20.988044 (+ 518us) client_negotiation.cc:769] Sending connection context
0812 00:39:20.988382 (+ 338us) client_negotiation.cc:240] Negotiation successful
0812 00:39:20.988985 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":386,"threads_started":1}
I20260812 00:39:20.991848 16012 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33029
I20260812 00:39:20.992228 16012 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:20.993176 16012 heartbeater.cc:511] Master 127.7.244.126:33029 requested a full tablet report, sending...
I20260812 00:39:20.995051 15291 ts_manager.cc:194] Registered new tserver with Master: b5bbb0070f3f41e686d276e92b9a218b (127.7.244.66:43289)
I20260812 00:39:20.996011 15291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.66@KRBTEST.COM'} at 127.7.244.66:51043
WARNING: no policy specified for kudu/127.7.244.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
Entry for principal kudu/127.7.244.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab.
I20260812 00:39:21.045961 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33029
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:40565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:21.416841 16021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:21.417208 16021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:21.417402 16021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:21.425464 16021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:21.425647 16021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:21.425849 16021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:39:21.436739 16021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:40565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.67.keytab
--principal=kudu/127.7.244.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:33029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:21.438246 16021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:21.439702 16021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 00:39:21.450714 15718 heartbeater.cc:503] Master 127.7.244.126:33029 was elected leader, sending a full tablet report...
W20260812 00:39:21.454921 16032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:21.454982 16036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:21.455118 16044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:21.455644 16021 server_base.cc:1034] running on GCE node
I20260812 00:39:21.456431 16021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:21.457518 16021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:21.458734 16021 hybrid_clock.cc:648] HybridClock initialized: now 1786495161458682 us; error 46 us; skew 500 ppm
I20260812 00:39:21.459064 16021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:21 dist-test-slave-gjw7 krb5kdc[15198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495161, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:21.462342 16021 init.cc:377] Logged in from keytab as kudu/127.7.244.67@KRBTEST.COM (short username kudu)
I20260812 00:39:21.463925 16021 webserver.cc:460] Webserver started at http://127.7.244.67:37515/ using document root <none> and password file <none>
I20260812 00:39:21.464663 16021 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:21.464792 16021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:21.465111 16021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:21.468273 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "df2ac45f29844bc1a37094db4fc4248b"
format_stamp: "Formatted at 2026-08-12 00:39:21 on dist-test-slave-gjw7"
server_key: "549a52ad6bb2b0350bae083f9831ae3a"
server_key_iv: "5fe47ab69d5b7809966f337602afc2fc"
server_key_version: "encryptionkey@0"
I20260812 00:39:21.469158 16021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "df2ac45f29844bc1a37094db4fc4248b"
format_stamp: "Formatted at 2026-08-12 00:39:21 on dist-test-slave-gjw7"
server_key: "549a52ad6bb2b0350bae083f9831ae3a"
server_key_iv: "5fe47ab69d5b7809966f337602afc2fc"
server_key_version: "encryptionkey@0"
I20260812 00:39:21.474974 16021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 00:39:21.478992 16056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:21.480690 16021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:39:21.480916 16021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "df2ac45f29844bc1a37094db4fc4248b"
format_stamp: "Formatted at 2026-08-12 00:39:21 on dist-test-slave-gjw7"
server_key: "549a52ad6bb2b0350bae083f9831ae3a"
server_key_iv: "5fe47ab69d5b7809966f337602afc2fc"
server_key_version: "encryptionkey@0"
I20260812 00:39:21.481145 16021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:21.506006 16021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:21.510915 16021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:21.511221 16021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:21.512413 16021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:21.514103 16021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:21.514191 16021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:21.514322 16021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:21.514377 16021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:21.547412 16021 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:37887
I20260812 00:39:21.547464 16311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:37887 every 8 connection(s)
I20260812 00:39:21.548736 16021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:39:21.558776 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 16021
I20260812 00:39:21.558990 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:39:21.559581 8145 external_mini_cluster.cc:1428] Setting key 7eb0788741989a1f21842215b21b8410
Aug 12 00:39:21 dist-test-slave-gjw7 krb5kdc[15198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495161, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.67@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:21.571645 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.551472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:45939 (local address 127.7.244.126:33029)
0812 00:39:21.551958 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.551963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.552830 (+ 867us) server_negotiation.cc:407] Connection header received
0812 00:39:21.554896 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.554902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.555000 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.555111 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:21.557104 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.557960 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.560356 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.560949 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.561092 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.565174 (+ 4082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:21.565179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:21.565183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:21.565222 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:21.567935 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:21.568882 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:21.568888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:21.568891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:21.569048 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:21.569715 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:21.569720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:21.569722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:21.570053 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:21.570205 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.571087 (+ 882us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.571332 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":247,"threads_started":1}
I20260812 00:39:21.572513 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.551897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:33029 (local address 127.7.244.67:45939)
0812 00:39:21.552644 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:21.552684 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:21.554554 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:21.555533 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:21.555543 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:21.556162 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:21.556883 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.556906 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.558140 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.558145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.560168 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.560184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.561249 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.561260 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.561618 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.562393 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:39:21.562419 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:21.564925 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:21.568132 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:21.568142 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:21.568159 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:21.568693 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:21.569244 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:39:21.569248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:39:21.569251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:39:21.569568 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:39:21.570182 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:21.570190 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:21.570607 (+ 417us) client_negotiation.cc:769] Sending connection context
0812 00:39:21.570859 (+ 252us) client_negotiation.cc:240] Negotiation successful
0812 00:39:21.571337 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":283,"threads_started":1}
I20260812 00:39:21.574254 16313 heartbeater.cc:347] Connected to a master server at 127.7.244.126:33029
I20260812 00:39:21.574632 16313 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:21.575635 16313 heartbeater.cc:511] Master 127.7.244.126:33029 requested a full tablet report, sending...
I20260812 00:39:21.577227 15291 ts_manager.cc:194] Registered new tserver with Master: df2ac45f29844bc1a37094db4fc4248b (127.7.244.67:37887)
I20260812 00:39:21.578125 15291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.244.67@KRBTEST.COM'} at 127.7.244.67:45939
I20260812 00:39:21.588289 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:39:21.605276 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.592589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39508 (local address 127.7.244.126:33029)
0812 00:39:21.592835 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.592842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.593151 (+ 309us) server_negotiation.cc:407] Connection header received
0812 00:39:21.593355 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.593359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.593447 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.593529 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:21.594933 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.595651 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.596937 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.597423 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.597529 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.600014 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:21.600020 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:21.600023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:21.600060 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:21.602552 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:21.603288 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:21.603293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:21.603296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:21.603454 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:21.603814 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:21.603818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:21.603820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:21.604227 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:21.604423 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.604785 (+ 362us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.604993 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 00:39:21.609599 15291 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:39:21.617472 15291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:39:21.642338 16343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.631523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.7.244.66:43289)
0812 00:39:21.632183 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.632191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.633413 (+ 1222us) server_negotiation.cc:407] Connection header received
0812 00:39:21.633663 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.633672 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.633944 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.634131 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:21.635187 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.636862 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.639807 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.640910 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.641046 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.641273 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.641776 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.641974 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":383,"threads_started":1}
I20260812 00:39:21.642575 16345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.631751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51664 (local address 127.7.244.67:37887)
0812 00:39:21.633952 (+ 2201us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.633960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.633993 (+ 33us) server_negotiation.cc:407] Connection header received
0812 00:39:21.634087 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.634094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.634302 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.634484 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:21.635463 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.637036 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.639894 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.641426 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.641587 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.641829 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.642036 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.642238 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1999,"thread_start_us":1848,"threads_started":1}
I20260812 00:39:21.642596 16342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.631203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:37887 (local address 127.0.0.1:51664)
0812 00:39:21.632678 (+ 1475us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:21.632721 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:21.632863 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:21.634777 (+ 1914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:21.634782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:21.634805 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:21.635260 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.635273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.637176 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.637184 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.639675 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.639689 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.641666 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.641673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.641849 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.641882 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 00:39:21.641991 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 00:39:21.642207 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"mutex_wait_us":86,"thread_start_us":1116,"threads_started":1}
I20260812 00:39:21.643445 16337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.631263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:43289 (local address 127.0.0.1:52998)
0812 00:39:21.633282 (+ 2019us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:21.633325 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:21.633455 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:21.634465 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:21.634473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:21.634504 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:21.634935 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.634953 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.637044 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.637050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.639640 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.639656 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.641116 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.641123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.641465 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.641520 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 00:39:21.642901 (+ 1381us) client_negotiation.cc:240] Negotiation successful
0812 00:39:21.643048 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1928,"mutex_wait_us":119,"thread_start_us":2444,"threads_started":1}
I20260812 00:39:21.645664 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.634094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55104)
0812 00:39:21.636167 (+ 2073us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:21.636183 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:21.636310 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:21.636818 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:21.636822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:21.636848 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:21.637319 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.637335 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.639286 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.639292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.641454 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.641469 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.644859 (+ 3390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.644863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.644991 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.645018 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 00:39:21.645123 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 00:39:21.645271 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1964,"mutex_wait_us":2,"thread_start_us":1820,"threads_started":1}
I20260812 00:39:21.645740 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.634723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.7.244.65:44015)
0812 00:39:21.635705 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.635725 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.636292 (+ 567us) server_negotiation.cc:407] Connection header received
0812 00:39:21.636468 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.636474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.636643 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.636826 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:21.637538 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.639116 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.641656 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.644415 (+ 2759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.644519 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.644690 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.645209 (+ 519us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.645432 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":655,"threads_started":1}
I20260812 00:39:21.647076 16161 tablet_service.cc:1467] Processing CreateTablet for tablet a5e9030698944256b08c893a6175c2c5 (DEFAULT_TABLE table=test-table [id=245d203500b14c979a6c20ffa5bddf27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:21.647985 15865 tablet_service.cc:1467] Processing CreateTablet for tablet a5e9030698944256b08c893a6175c2c5 (DEFAULT_TABLE table=test-table [id=245d203500b14c979a6c20ffa5bddf27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:21.649613 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e9030698944256b08c893a6175c2c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:21.649632 16161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e9030698944256b08c893a6175c2c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:21.649901 15568 tablet_service.cc:1467] Processing CreateTablet for tablet a5e9030698944256b08c893a6175c2c5 (DEFAULT_TABLE table=test-table [id=245d203500b14c979a6c20ffa5bddf27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:21.651479 15568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e9030698944256b08c893a6175c2c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:21.663626 16357 tablet_bootstrap.cc:492] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Bootstrap starting.
I20260812 00:39:21.664932 16359 tablet_bootstrap.cc:492] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Bootstrap starting.
I20260812 00:39:21.666621 16360 tablet_bootstrap.cc:492] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3: Bootstrap starting.
I20260812 00:39:21.667959 16357 tablet_bootstrap.cc:654] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:21.668686 16359 tablet_bootstrap.cc:654] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:21.669377 16357 log.cc:824] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:21.669647 16360 tablet_bootstrap.cc:654] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:21.670074 16359 log.cc:824] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:21.670831 16360 log.cc:824] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:21.672968 16357 tablet_bootstrap.cc:492] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: No bootstrap required, opened a new log
I20260812 00:39:21.673336 16357 ts_tablet_manager.cc:1397] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260812 00:39:21.674322 16359 tablet_bootstrap.cc:492] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: No bootstrap required, opened a new log
I20260812 00:39:21.674630 16359 ts_tablet_manager.cc:1397] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260812 00:39:21.675137 16360 tablet_bootstrap.cc:492] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3: No bootstrap required, opened a new log
I20260812 00:39:21.675439 16360 ts_tablet_manager.cc:1397] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260812 00:39:21.680815 16357 raft_consensus.cc:348] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.681231 16357 raft_consensus.cc:374] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:21.681416 16357 raft_consensus.cc:729] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5bbb0070f3f41e686d276e92b9a218b, State: Initialized, Role: FOLLOWER
I20260812 00:39:21.682078 16357 consensus_queue.cc:260] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.684311 16012 heartbeater.cc:503] Master 127.7.244.126:33029 was elected leader, sending a full tablet report...
I20260812 00:39:21.685379 16359 raft_consensus.cc:348] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.685953 16359 raft_consensus.cc:374] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:21.686105 16359 raft_consensus.cc:729] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df2ac45f29844bc1a37094db4fc4248b, State: Initialized, Role: FOLLOWER
I20260812 00:39:21.686154 16357 ts_tablet_manager.cc:1428] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Time spent starting tablet: real 0.013s user 0.006s sys 0.006s
I20260812 00:39:21.686184 16360 raft_consensus.cc:348] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.686688 16360 raft_consensus.cc:374] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:21.686792 16360 raft_consensus.cc:729] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b1d4dd44d3e401a9532b89ca809dab3, State: Initialized, Role: FOLLOWER
I20260812 00:39:21.686987 16359 consensus_queue.cc:260] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.687386 16360 consensus_queue.cc:260] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.688174 16313 heartbeater.cc:503] Master 127.7.244.126:33029 was elected leader, sending a full tablet report...
I20260812 00:39:21.688930 16359 ts_tablet_manager.cc:1428] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260812 00:39:21.690629 16360 ts_tablet_manager.cc:1428] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260812 00:39:21.711469 16365 raft_consensus.cc:482] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:39:21.711851 16365 raft_consensus.cc:504] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.713836 16365 leader_election.cc:288] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df2ac45f29844bc1a37094db4fc4248b (127.7.244.67:37887), 5b1d4dd44d3e401a9532b89ca809dab3 (127.7.244.65:44015)
W20260812 00:39:21.717631 16014 tablet.cc:2369] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:21.721098 16345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.714268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:55961 (local address 127.7.244.67:37887)
0812 00:39:21.714775 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.714782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.714966 (+ 184us) server_negotiation.cc:407] Connection header received
0812 00:39:21.715257 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.715261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.715319 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.715404 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:21.716068 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.716826 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.719199 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.719970 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.720132 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.720212 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.720708 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.720847 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312}
I20260812 00:39:21.721148 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.714505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:39335 (local address 127.7.244.65:44015)
0812 00:39:21.714956 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.714962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.715163 (+ 201us) server_negotiation.cc:407] Connection header received
0812 00:39:21.715378 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.715384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.715448 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.715526 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:21.716195 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.716916 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.719123 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.719961 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.720121 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.720208 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.720749 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.720900 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302}
I20260812 00:39:21.721225 16377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.714033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:37887 (local address 127.7.244.66:55961)
0812 00:39:21.714834 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:21.714858 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:21.715104 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:21.715425 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:21.715429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:21.715452 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:21.715899 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.715913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.716967 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.716972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.718980 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.718993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.720253 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.720260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.720504 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.720570 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 00:39:21.720660 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 00:39:21.720804 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":90,"spinlock_wait_cycles":20224,"thread_start_us":375,"threads_started":1}
I20260812 00:39:21.722011 16379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.714223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.7.244.66:39335)
0812 00:39:21.715068 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:21.715083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:21.715208 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:21.715562 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:21.715567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:21.715609 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:21.716052 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.716062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.717061 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.717066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.718923 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:21.718943 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.720216 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.720223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:21.720501 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.720571 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 00:39:21.721441 (+ 870us) client_negotiation.cc:240] Negotiation successful
0812 00:39:21.721581 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":66,"thread_start_us":495,"threads_started":1}
I20260812 00:39:21.723093 16214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9030698944256b08c893a6175c2c5" candidate_uuid: "b5bbb0070f3f41e686d276e92b9a218b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2ac45f29844bc1a37094db4fc4248b" is_pre_election: true
I20260812 00:39:21.723342 15615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9030698944256b08c893a6175c2c5" candidate_uuid: "b5bbb0070f3f41e686d276e92b9a218b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" is_pre_election: true
I20260812 00:39:21.723775 16214 raft_consensus.cc:2454] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bbb0070f3f41e686d276e92b9a218b in term 0.
I20260812 00:39:21.723886 15615 raft_consensus.cc:2454] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5bbb0070f3f41e686d276e92b9a218b in term 0.
I20260812 00:39:21.724582 15772 leader_election.cc:302] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b1d4dd44d3e401a9532b89ca809dab3, b5bbb0070f3f41e686d276e92b9a218b; no voters:
I20260812 00:39:21.724994 16365 raft_consensus.cc:2781] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:39:21.725162 16365 raft_consensus.cc:482] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:39:21.725265 16365 raft_consensus.cc:3037] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:21.727555 16365 raft_consensus.cc:504] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.728230 16365 leader_election.cc:288] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [CANDIDATE]: Term 1 election: Requested vote from peers df2ac45f29844bc1a37094db4fc4248b (127.7.244.67:37887), 5b1d4dd44d3e401a9532b89ca809dab3 (127.7.244.65:44015)
I20260812 00:39:21.728849 15615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9030698944256b08c893a6175c2c5" candidate_uuid: "b5bbb0070f3f41e686d276e92b9a218b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b1d4dd44d3e401a9532b89ca809dab3"
I20260812 00:39:21.728849 16214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e9030698944256b08c893a6175c2c5" candidate_uuid: "b5bbb0070f3f41e686d276e92b9a218b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df2ac45f29844bc1a37094db4fc4248b"
I20260812 00:39:21.729103 15615 raft_consensus.cc:3037] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:21.729103 16214 raft_consensus.cc:3037] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:21.732535 15615 raft_consensus.cc:2454] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bbb0070f3f41e686d276e92b9a218b in term 1.
I20260812 00:39:21.732535 16214 raft_consensus.cc:2454] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5bbb0070f3f41e686d276e92b9a218b in term 1.
I20260812 00:39:21.733052 15774 leader_election.cc:302] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5bbb0070f3f41e686d276e92b9a218b, df2ac45f29844bc1a37094db4fc4248b; no voters:
I20260812 00:39:21.733433 16365 raft_consensus.cc:2781] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:21.733930 16365 raft_consensus.cc:686] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 1 LEADER]: Becoming Leader. State: Replica: b5bbb0070f3f41e686d276e92b9a218b, State: Running, Role: LEADER
I20260812 00:39:21.734503 16365 consensus_queue.cc:237] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } }
I20260812 00:39:21.739457 15289 catalog_manager.cc:5495] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b reported cstate change: term changed from 0 to 1, leader changed from <none> to b5bbb0070f3f41e686d276e92b9a218b (127.7.244.66). New cstate: current_term: 1 leader_uuid: "b5bbb0070f3f41e686d276e92b9a218b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5bbb0070f3f41e686d276e92b9a218b" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 43289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 } health_report { overall_health: UNKNOWN } } }
I20260812 00:39:21.771519 16343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.765583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.7.244.66:43289)
0812 00:39:21.765812 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.765819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.765953 (+ 134us) server_negotiation.cc:407] Connection header received
0812 00:39:21.766203 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.766207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.766316 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.766467 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:39:21.766914 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.767660 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.769112 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.769558 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.769674 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.770245 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:39:21.770830 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:39:21.770978 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.771124 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.771239 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:39:21.780402 15615 raft_consensus.cc:1261] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 1 FOLLOWER]: Refusing update from remote peer b5bbb0070f3f41e686d276e92b9a218b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:21.780378 16214 raft_consensus.cc:1261] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 1 FOLLOWER]: Refusing update from remote peer b5bbb0070f3f41e686d276e92b9a218b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:21.781615 16380 consensus_queue.cc:1035] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [LEADER]: Connected to new peer: Peer: permanent_uuid: "df2ac45f29844bc1a37094db4fc4248b" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 37887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:21.781991 16365 consensus_queue.cc:1035] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b1d4dd44d3e401a9532b89ca809dab3" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 44015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:21.793535 16387 mvcc.cc:205] Tried to move back new op lower bound from 7317484182642106368 to 7317484182468763648. Current Snapshot: MvccSnapshot[applied={T|T < 7317484182642106368}]
W20260812 00:39:21.800390 16315 tablet.cc:2369] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:21.812875 15289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:21.813230 15289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:21.817520 15289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04b3caede9204fbd88a381f2bae111a7: Sending DeleteTablet for 3 replicas of tablet a5e9030698944256b08c893a6175c2c5
I20260812 00:39:21.818789 16161 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e9030698944256b08c893a6175c2c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:21 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:51664
I20260812 00:39:21.818935 15568 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e9030698944256b08c893a6175c2c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:21 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:55104
I20260812 00:39:21.820046 15865 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e9030698944256b08c893a6175c2c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:21 UTC) from {username='kudu', principal='kudu/127.7.244.126@KRBTEST.COM'} at 127.0.0.1:52998
I20260812 00:39:21.820345 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 15434
I20260812 00:39:21.838907 16418 tablet_replica.cc:330] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: stopping tablet replica
I20260812 00:39:21.839515 16418 raft_consensus.cc:2229] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:39:21.840287 16418 raft_consensus.cc:2258] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:21.841106 16417 tablet_replica.cc:330] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3: stopping tablet replica
I20260812 00:39:21.841575 16417 raft_consensus.cc:2229] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:21.842155 16417 raft_consensus.cc:2258] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:21.843231 16412 tablet_replica.cc:330] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: stopping tablet replica
I20260812 00:39:21.843662 16412 raft_consensus.cc:2229] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:21.844195 16412 raft_consensus.cc:2258] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 12 00:39:21 dist-test-slave-gjw7 krb5kdc[15198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.65@KRBTEST.COM
I20260812 00:39:21.846122 16418 ts_tablet_manager.cc:1905] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:21.851032 16417 ts_tablet_manager.cc:1905] T a5e9030698944256b08c893a6175c2c5 P 5b1d4dd44d3e401a9532b89ca809dab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:21.853525 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:21.821959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.7.244.65:44015)
0812 00:39:21.822166 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:21.822173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:21.832651 (+ 10478us) server_negotiation.cc:407] Connection header received
0812 00:39:21.833277 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:21.833285 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:21.833450 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:21.833536 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:21.835606 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.836978 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.839281 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:21.840042 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:21.840153 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:21.846073 (+ 5920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:21.846088 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:21.846124 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:21.846186 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:21.849157 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:21.850251 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:21.850263 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:21.850267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:21.850464 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:21.851249 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:21.851255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:21.851259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:21.851799 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:21.852135 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:21.852801 (+ 666us) server_negotiation.cc:299] Negotiation successful
0812 00:39:21.853071 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 00:39:21.854341 16418 ts_tablet_manager.cc:1918] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:21.854560 16418 log.cc:1196] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/a5e9030698944256b08c893a6175c2c5
I20260812 00:39:21.855172 16418 ts_tablet_manager.cc:1939] T a5e9030698944256b08c893a6175c2c5 P b5bbb0070f3f41e686d276e92b9a218b: Deleting consensus metadata
I20260812 00:39:21.855705 15705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:21.856205 16412 ts_tablet_manager.cc:1905] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:21.857241 15262 catalog_manager.cc:4841] TS b5bbb0070f3f41e686d276e92b9a218b (127.7.244.66:43289): tablet a5e9030698944256b08c893a6175c2c5 (table test-table [id=245d203500b14c979a6c20ffa5bddf27]) successfully deleted
I20260812 00:39:21.863529 16412 ts_tablet_manager.cc:1918] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:21.863744 16412 log.cc:1196] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/a5e9030698944256b08c893a6175c2c5
I20260812 00:39:21.864214 16412 ts_tablet_manager.cc:1939] T a5e9030698944256b08c893a6175c2c5 P df2ac45f29844bc1a37094db4fc4248b: Deleting consensus metadata
I20260812 00:39:21.866389 15266 catalog_manager.cc:4841] TS df2ac45f29844bc1a37094db4fc4248b (127.7.244.67:37887): tablet a5e9030698944256b08c893a6175c2c5 (table test-table [id=245d203500b14c979a6c20ffa5bddf27]) successfully deleted
I20260812 00:39:21.985565 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 15434
I20260812 00:39:22.025156 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 15728
W20260812 00:39:22.025063 15262 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.244.65:44015 (error 108)
I20260812 00:39:22.027413 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.026843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55124)
0812 00:39:22.027063 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:22.027206 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260812 00:39:22.027767 15262 catalog_manager.cc:4584] TS 5b1d4dd44d3e401a9532b89ca809dab3 (127.7.244.65:44015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5e9030698944256b08c893a6175c2c5: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Aug 12 00:39:22 dist-test-slave-gjw7 krb5kdc[15198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.66@KRBTEST.COM
I20260812 00:39:22.042863 16343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.026566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.7.244.66:43289)
0812 00:39:22.026788 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:22.026794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:22.026813 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:39:22.026897 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:22.026903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:22.027006 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:22.027106 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:22.028591 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:22.029632 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:22.030990 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:22.031461 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:22.031584 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:22.036431 (+ 4847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:22.036451 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:22.036472 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:22.036526 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:22.039738 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:22.040720 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:22.040738 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:22.040741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:22.040945 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:22.041413 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:22.041417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:22.041419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:22.041800 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:22.042084 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:22.042379 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 00:39:22.042574 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 00:39:22.043665 16002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:22.081771 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.081067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55126)
0812 00:39:22.081442 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:22.081581 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260812 00:39:22.082967 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.082571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55134)
0812 00:39:22.082735 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:22.082843 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 00:39:22.144204 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.143568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55148)
0812 00:39:22.143872 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:22.144012 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 00:39:22.145351 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.144911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55162)
0812 00:39:22.145072 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:22.145158 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 00:39:22.160320 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 15728
I20260812 00:39:22.190311 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 16021
Aug 12 00:39:22 dist-test-slave-gjw7 krb5kdc[15198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.67@KRBTEST.COM
I20260812 00:39:22.208218 16345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.191162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.7.244.67:37887)
0812 00:39:22.191355 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:22.191359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:22.191377 (+ 18us) server_negotiation.cc:407] Connection header received
0812 00:39:22.191436 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:22.191440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:22.191504 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:22.191585 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:22.192941 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:22.193838 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:22.195369 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:22.195808 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:22.195942 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:22.201414 (+ 5472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:22.201437 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:22.201456 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:22.201509 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:22.204645 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:22.205797 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:22.205807 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:22.205810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:22.206012 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:22.206506 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:22.206510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:22.206512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:22.207026 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:22.207283 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:22.207711 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 00:39:22.207929 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 00:39:22.208920 16295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:22.226806 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.226239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55170)
0812 00:39:22.226487 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:22.226588 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260812 00:39:22.227945 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.227519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55182)
0812 00:39:22.227728 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:22.227817 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 00:39:22.300521 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 16021
I20260812 00:39:22.333004 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 15223
I20260812 00:39:22.334441 15363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:22.446965 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:22.446371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:44015 (local address 127.0.0.1:55184)
0812 00:39:22.446603 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:22.446726 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.65:44015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260812 00:39:22.446934 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 15223
2026-08-12T00:39:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4856 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T00:39:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:22Z Disabled control of system clock
I20260812 00:39:22.551278 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:43621
--webserver_interface=127.7.244.126
--webserver_port=0
--builtin_ntp_servers=127.7.244.84:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:43621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:39:22.931942 16434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:22.932314 16434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:22.932438 16434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:22.940331 16434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:22.940488 16434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:22.940537 16434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:22.940629 16434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:22.940711 16434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:22.950760 16434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:43621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:43621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:22.952247 16434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:22.954427 16434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:22.967849 16446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:22.967808 16452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:22.967872 16448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:22.969640 16434 server_base.cc:1034] running on GCE node
I20260812 00:39:22.970878 16434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:22.972764 16434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:22.974056 16434 hybrid_clock.cc:648] HybridClock initialized: now 1786495162973992 us; error 55 us; skew 500 ppm
I20260812 00:39:22.974498 16434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:22.977427 16434 webserver.cc:460] Webserver started at http://127.7.244.126:35165/ using document root <none> and password file <none>
I20260812 00:39:22.978322 16434 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:22.978471 16434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:22.978863 16434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:22.982445 16434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "ba6940a706d146d8a6e273043664f88d"
format_stamp: "Formatted at 2026-08-12 00:39:22 on dist-test-slave-gjw7"
server_key: "1b756b3e832bb8c8ca8561f35c5af1b6"
server_key_iv: "0a4358b9ae35e51543d611b858346a81"
server_key_version: "encryptionkey@0"
I20260812 00:39:22.983455 16434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "ba6940a706d146d8a6e273043664f88d"
format_stamp: "Formatted at 2026-08-12 00:39:22 on dist-test-slave-gjw7"
server_key: "1b756b3e832bb8c8ca8561f35c5af1b6"
server_key_iv: "0a4358b9ae35e51543d611b858346a81"
server_key_version: "encryptionkey@0"
I20260812 00:39:22.990192 16434 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:39:22.994623 16461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:22.996258 16434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.004s
I20260812 00:39:22.996493 16434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "ba6940a706d146d8a6e273043664f88d"
format_stamp: "Formatted at 2026-08-12 00:39:22 on dist-test-slave-gjw7"
server_key: "1b756b3e832bb8c8ca8561f35c5af1b6"
server_key_iv: "0a4358b9ae35e51543d611b858346a81"
server_key_version: "encryptionkey@0"
I20260812 00:39:22.996737 16434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:23.018113 16434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:23.019093 16434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:23.019418 16434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:23.039187 16434 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:43621
I20260812 00:39:23.039220 16561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:43621 every 8 connection(s)
I20260812 00:39:23.040875 16434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:23.045109 16562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:23.045184 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 16434
I20260812 00:39:23.045389 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:23.045858 8145 external_mini_cluster.cc:1428] Setting key 315f4114a90192e2e0af4bd97670db9c
I20260812 00:39:23.060954 16562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d: Bootstrap starting.
I20260812 00:39:23.063653 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:23.047967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59938 (local address 127.7.244.126:43621)
0812 00:39:23.049241 (+ 1274us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:23.049257 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:23.049355 (+ 98us) server_negotiation.cc:407] Connection header received
0812 00:39:23.052195 (+ 2840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:23.052234 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:23.052998 (+ 764us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:23.053540 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:23.054133 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:23.056121 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:23.057755 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:23.058942 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:23.059134 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:23.060366 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:23.060379 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:23.060406 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:23.061198 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:23.061357 (+ 159us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:23.061390 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:23.061520 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:23.061813 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 00:39:23.062206 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":410,"threads_started":1}
I20260812 00:39:23.065194 16562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:23.066397 16562 log.cc:824] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:23.069577 16562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d: No bootstrap required, opened a new log
I20260812 00:39:23.077696 16562 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6940a706d146d8a6e273043664f88d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 43621 } }
I20260812 00:39:23.078172 16562 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:23.078289 16562 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6940a706d146d8a6e273043664f88d, State: Initialized, Role: FOLLOWER
I20260812 00:39:23.079017 16562 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6940a706d146d8a6e273043664f88d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 43621 } }
I20260812 00:39:23.079304 16562 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:23.079470 16562 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:23.079638 16562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:23.082038 16562 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6940a706d146d8a6e273043664f88d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 43621 } }
I20260812 00:39:23.082612 16562 leader_election.cc:302] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba6940a706d146d8a6e273043664f88d; no voters:
I20260812 00:39:23.083283 16562 leader_election.cc:288] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:23.083592 16575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:23.084326 16575 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [term 1 LEADER]: Becoming Leader. State: Replica: ba6940a706d146d8a6e273043664f88d, State: Running, Role: LEADER
I20260812 00:39:23.084946 16575 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6940a706d146d8a6e273043664f88d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 43621 } }
I20260812 00:39:23.085616 16562 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:23.088734 16580 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba6940a706d146d8a6e273043664f88d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6940a706d146d8a6e273043664f88d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 43621 } } }
I20260812 00:39:23.088738 16584 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba6940a706d146d8a6e273043664f88d. Latest consensus state: current_term: 1 leader_uuid: "ba6940a706d146d8a6e273043664f88d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba6940a706d146d8a6e273043664f88d" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 43621 } } }
I20260812 00:39:23.089567 16580 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:23.089570 16584 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:23.092540 16593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:23.098363 16593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:23.107697 16593 catalog_manager.cc:1347] Generated new cluster ID: 0734d0d9bb6046a784bf464d6a6fa8eb
I20260812 00:39:23.107836 16593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:23.129320 16593 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:23.130762 16593 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:23.143193 16593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d: Generated new TSK 0
I20260812 00:39:23.144219 16593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:23.150970 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.65:0
--local_ip_for_outbound_sockets=127.7.244.65
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:43621
--builtin_ntp_servers=127.7.244.84:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 00:39:23.553714 16611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:23.554041 16611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:23.554131 16611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:23.561683 16611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:23.561818 16611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:23.561931 16611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.65
I20260812 00:39:23.572414 16611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.244.65
--webserver_port=0
--tserver_master_addrs=127.7.244.126:43621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.65
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:23.573848 16611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:23.575786 16611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:23.590382 16628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:23.590713 16622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:23.590736 16621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:23.591332 16611 server_base.cc:1034] running on GCE node
I20260812 00:39:23.592064 16611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:23.593114 16611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:23.594369 16611 hybrid_clock.cc:648] HybridClock initialized: now 1786495163594291 us; error 65 us; skew 500 ppm
I20260812 00:39:23.594766 16611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:23.597270 16611 webserver.cc:460] Webserver started at http://127.7.244.65:40811/ using document root <none> and password file <none>
I20260812 00:39:23.598104 16611 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:23.598224 16611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:23.598583 16611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:23.602067 16611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/data/instance:
uuid: "9914fd29217d4e98bf184b19f6d6ab29"
format_stamp: "Formatted at 2026-08-12 00:39:23 on dist-test-slave-gjw7"
server_key: "c6f93efff5abfe4e0e13ec2c6fea2c34"
server_key_iv: "3a39739bae886440d29de3bc0e19a061"
server_key_version: "encryptionkey@0"
I20260812 00:39:23.602998 16611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance:
uuid: "9914fd29217d4e98bf184b19f6d6ab29"
format_stamp: "Formatted at 2026-08-12 00:39:23 on dist-test-slave-gjw7"
server_key: "c6f93efff5abfe4e0e13ec2c6fea2c34"
server_key_iv: "3a39739bae886440d29de3bc0e19a061"
server_key_version: "encryptionkey@0"
I20260812 00:39:23.609236 16611 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 00:39:23.613530 16634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:23.615111 16611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:39:23.615293 16611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/wal
uuid: "9914fd29217d4e98bf184b19f6d6ab29"
format_stamp: "Formatted at 2026-08-12 00:39:23 on dist-test-slave-gjw7"
server_key: "c6f93efff5abfe4e0e13ec2c6fea2c34"
server_key_iv: "3a39739bae886440d29de3bc0e19a061"
server_key_version: "encryptionkey@0"
I20260812 00:39:23.615551 16611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:23.633756 16611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:23.634706 16611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:23.635039 16611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:23.636148 16611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:23.637812 16611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:23.637921 16611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:23.638034 16611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:23.638106 16611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:23.670226 16611 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.65:38301
I20260812 00:39:23.670325 16856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.65:38301 every 8 connection(s)
I20260812 00:39:23.671698 16611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/data/info.pb
I20260812 00:39:23.680814 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 16611
I20260812 00:39:23.680995 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/wal/instance
I20260812 00:39:23.681473 8145 external_mini_cluster.cc:1428] Setting key ecd314d5df81d4642439c60645c0061e
I20260812 00:39:23.684608 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.66:0
--local_ip_for_outbound_sockets=127.7.244.66
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:43621
--builtin_ntp_servers=127.7.244.84:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:39:23.685603 16861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:23.674511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:55273 (local address 127.7.244.126:43621)
0812 00:39:23.675018 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:23.675025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:23.676263 (+ 1238us) server_negotiation.cc:407] Connection header received
0812 00:39:23.678208 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:23.678213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:23.678299 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:23.678405 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:23.679598 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:23.680308 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:23.682254 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:23.682893 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:23.683014 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:23.684623 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:23.684628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:23.684631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:23.684650 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:23.684704 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:23.684720 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:23.684811 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:23.685088 (+ 277us) server_negotiation.cc:299] Negotiation successful
0812 00:39:23.685195 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":372,"thread_start_us":238,"threads_started":1}
I20260812 00:39:23.686722 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:23.675102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:43621 (local address 127.7.244.65:55273)
0812 00:39:23.676085 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:23.676132 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:23.677891 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:23.678666 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:23.678676 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:23.678791 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:23.679439 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:23.679459 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:23.680433 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:23.680436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:23.682025 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:23.682042 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:23.683154 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:23.683162 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:23.683468 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:23.684248 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:23.684251 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:23.684297 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:23.684404 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:23.684864 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:23.684877 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:23.684891 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 00:39:23.685057 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 00:39:23.685545 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":289,"threads_started":1}
I20260812 00:39:23.688367 16859 heartbeater.cc:347] Connected to a master server at 127.7.244.126:43621
I20260812 00:39:23.688690 16859 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:23.689553 16859 heartbeater.cc:511] Master 127.7.244.126:43621 requested a full tablet report, sending...
I20260812 00:39:23.691882 16512 ts_manager.cc:194] Registered new tserver with Master: 9914fd29217d4e98bf184b19f6d6ab29 (127.7.244.65:38301)
I20260812 00:39:23.693506 16512 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.65:55273
W20260812 00:39:24.040215 16866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:24.040539 16866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:24.040655 16866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:24.048158 16866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:24.048346 16866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:24.048522 16866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.66
I20260812 00:39:24.058393 16866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.244.66
--webserver_port=0
--tserver_master_addrs=127.7.244.126:43621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.66
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:24.059870 16866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:24.061959 16866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:24.075834 16884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:24.075834 16886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:24.076007 16894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:24.077163 16866 server_base.cc:1034] running on GCE node
I20260812 00:39:24.078016 16866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:24.079072 16866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:24.080348 16866 hybrid_clock.cc:648] HybridClock initialized: now 1786495164080243 us; error 62 us; skew 500 ppm
I20260812 00:39:24.080717 16866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:24.083326 16866 webserver.cc:460] Webserver started at http://127.7.244.66:34141/ using document root <none> and password file <none>
I20260812 00:39:24.084060 16866 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:24.084162 16866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:24.084504 16866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:24.087754 16866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/data/instance:
uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328"
format_stamp: "Formatted at 2026-08-12 00:39:24 on dist-test-slave-gjw7"
server_key: "36575a268ebb03016b5f235353612abd"
server_key_iv: "7e5f9e06b98a8af34de3ed959e483df5"
server_key_version: "encryptionkey@0"
I20260812 00:39:24.088681 16866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance:
uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328"
format_stamp: "Formatted at 2026-08-12 00:39:24 on dist-test-slave-gjw7"
server_key: "36575a268ebb03016b5f235353612abd"
server_key_iv: "7e5f9e06b98a8af34de3ed959e483df5"
server_key_version: "encryptionkey@0"
I20260812 00:39:24.094926 16866 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 00:39:24.099160 16909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:24.100833 16866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 00:39:24.101033 16866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/wal
uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328"
format_stamp: "Formatted at 2026-08-12 00:39:24 on dist-test-slave-gjw7"
server_key: "36575a268ebb03016b5f235353612abd"
server_key_iv: "7e5f9e06b98a8af34de3ed959e483df5"
server_key_version: "encryptionkey@0"
I20260812 00:39:24.101275 16866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:24.144189 16866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:24.145123 16866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:24.145465 16866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:24.146585 16866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:24.148325 16866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:24.148442 16866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:24.148591 16866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:24.148660 16866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:24.184213 16866 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.66:44121
I20260812 00:39:24.184253 17140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.66:44121 every 8 connection(s)
I20260812 00:39:24.185715 16866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/data/info.pb
I20260812 00:39:24.188534 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 16866
I20260812 00:39:24.188740 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/wal/instance
I20260812 00:39:24.189373 8145 external_mini_cluster.cc:1428] Setting key 1c7d700ca491292b41750979794b0097
I20260812 00:39:24.193162 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.244.67:0
--local_ip_for_outbound_sockets=127.7.244.67
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:43621
--builtin_ntp_servers=127.7.244.84:34317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 00:39:24.203045 17148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.188877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.66:39279 (local address 127.7.244.126:43621)
0812 00:39:24.189417 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.189423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.191069 (+ 1646us) server_negotiation.cc:407] Connection header received
0812 00:39:24.193737 (+ 2668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.193742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.193831 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.193933 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:24.195753 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.196721 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.199271 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.199817 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.199955 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.201920 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:24.201927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:24.201932 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:24.201957 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:24.202020 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:24.202036 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:24.202122 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.202599 (+ 477us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.202714 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":239,"threads_started":1}
I20260812 00:39:24.204499 17152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.189602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:43621 (local address 127.7.244.66:39279)
0812 00:39:24.190884 (+ 1282us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:24.190938 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:24.193357 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:24.194295 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:24.194310 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:24.194464 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:24.195449 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.195495 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.196941 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.196948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.199064 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.199086 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.200105 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.200114 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.200487 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.201422 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:24.201426 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:24.201498 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:24.201620 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:24.202208 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:24.202221 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:24.202230 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 00:39:24.202470 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 00:39:24.202957 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":343,"threads_started":1}
I20260812 00:39:24.206298 17141 heartbeater.cc:347] Connected to a master server at 127.7.244.126:43621
I20260812 00:39:24.206743 17141 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:24.207823 17141 heartbeater.cc:511] Master 127.7.244.126:43621 requested a full tablet report, sending...
I20260812 00:39:24.209688 16512 ts_manager.cc:194] Registered new tserver with Master: 3f62f63c0c5c4d70bfcb4fa0d10d8328 (127.7.244.66:44121)
I20260812 00:39:24.210584 16512 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.66:39279
W20260812 00:39:24.556309 17155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:24.556636 17155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:24.556736 17155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:24.564234 17155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:24.564361 17155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:24.564498 17155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.244.67
I20260812 00:39:24.574065 17155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.244.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.244.67
--webserver_port=0
--tserver_master_addrs=127.7.244.126:43621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.244.67
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:24.575476 17155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:24.577482 17155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:24.591379 17163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:24.591372 17166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:24.591372 17171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:24.592500 17155 server_base.cc:1034] running on GCE node
I20260812 00:39:24.593194 17155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:24.594272 17155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:24.595505 17155 hybrid_clock.cc:648] HybridClock initialized: now 1786495164595448 us; error 45 us; skew 500 ppm
I20260812 00:39:24.595847 17155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 00:39:24.598548 17155 webserver.cc:460] Webserver started at http://127.7.244.67:37377/ using document root <none> and password file <none>
I20260812 00:39:24.599310 17155 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:24.599416 17155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:24.599761 17155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:24.602948 17155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/data/instance:
uuid: "414ce8a89f384a9ba01551d14eb5ec93"
format_stamp: "Formatted at 2026-08-12 00:39:24 on dist-test-slave-gjw7"
server_key: "0405b5965ea51b8fca836f2c686916db"
server_key_iv: "14805293f8d7d5bd8b380e18bf50f9b3"
server_key_version: "encryptionkey@0"
I20260812 00:39:24.603821 17155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance:
uuid: "414ce8a89f384a9ba01551d14eb5ec93"
format_stamp: "Formatted at 2026-08-12 00:39:24 on dist-test-slave-gjw7"
server_key: "0405b5965ea51b8fca836f2c686916db"
server_key_iv: "14805293f8d7d5bd8b380e18bf50f9b3"
server_key_version: "encryptionkey@0"
I20260812 00:39:24.609843 17155 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 00:39:24.613886 17183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:24.615404 17155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:39:24.615577 17155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/wal
uuid: "414ce8a89f384a9ba01551d14eb5ec93"
format_stamp: "Formatted at 2026-08-12 00:39:24 on dist-test-slave-gjw7"
server_key: "0405b5965ea51b8fca836f2c686916db"
server_key_iv: "14805293f8d7d5bd8b380e18bf50f9b3"
server_key_version: "encryptionkey@0"
I20260812 00:39:24.615818 17155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:24.641839 17155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:24.642768 17155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:24.643090 17155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:24.644181 17155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 00:39:24.645942 17155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 00:39:24.646050 17155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:24.646165 17155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 00:39:24.646242 17155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:24.678737 17155 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.67:44581
I20260812 00:39:24.678817 17410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.67:44581 every 8 connection(s)
I20260812 00:39:24.680023 17155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/data/info.pb
I20260812 00:39:24.690531 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 17155
I20260812 00:39:24.690726 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/wal/instance
I20260812 00:39:24.691313 8145 external_mini_cluster.cc:1428] Setting key 2e2f9fbc748f31a5e0a9450642433cf1
I20260812 00:39:24.697058 17148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.682660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.67:36837 (local address 127.7.244.126:43621)
0812 00:39:24.682898 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.682905 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.684271 (+ 1366us) server_negotiation.cc:407] Connection header received
0812 00:39:24.686183 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.686187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.686263 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.686353 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:24.687590 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.688395 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.690216 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.690637 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.690770 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.696259 (+ 5489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:24.696264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:24.696267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:24.696287 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:24.696325 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:24.696338 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:24.696417 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.696645 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.696748 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 00:39:24.697193 16859 heartbeater.cc:503] Master 127.7.244.126:43621 was elected leader, sending a full tablet report...
I20260812 00:39:24.698417 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.683143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:43621 (local address 127.7.244.67:36837)
0812 00:39:24.684098 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:24.684143 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:24.685895 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:24.686618 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:24.686628 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:24.686752 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:39:24.687417 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.687439 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.688545 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.688549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.690017 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.690054 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.690917 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.690927 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.691260 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.695843 (+ 4583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 00:39:24.695847 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:39:24.695905 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 00:39:24.696032 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:39:24.696440 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:39:24.696452 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:39:24.696460 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 00:39:24.696622 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 00:39:24.697024 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":275,"threads_started":1}
I20260812 00:39:24.700099 17411 heartbeater.cc:347] Connected to a master server at 127.7.244.126:43621
I20260812 00:39:24.700446 17411 heartbeater.cc:464] Registering TS with master...
I20260812 00:39:24.701475 17411 heartbeater.cc:511] Master 127.7.244.126:43621 requested a full tablet report, sending...
I20260812 00:39:24.703132 16512 ts_manager.cc:194] Registered new tserver with Master: 414ce8a89f384a9ba01551d14eb5ec93 (127.7.244.67:44581)
I20260812 00:39:24.703994 16512 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.244.67:36837
I20260812 00:39:24.707019 8145 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 00:39:24.717233 17148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.711352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59950 (local address 127.7.244.126:43621)
0812 00:39:24.711582 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.711589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.712009 (+ 420us) server_negotiation.cc:407] Connection header received
0812 00:39:24.712178 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.712182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.712254 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.712334 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:24.712828 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.713625 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.714917 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.715352 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.715452 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.716445 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:24.716451 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:24.716454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:24.716474 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:24.716508 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:24.716519 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:24.716588 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.716822 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.716943 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:39:24.721544 16512 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 00:39:24.728642 16512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 00:39:24.754478 17445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.742895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:44581 (local address 127.0.0.1:36440)
0812 00:39:24.744466 (+ 1571us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:24.744484 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:24.744588 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:24.745479 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:24.745487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:24.745521 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:24.745967 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.745981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.748434 (+ 2453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.748440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.751272 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.751283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.753205 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.753212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.753497 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.753581 (+ 84us) client_negotiation.cc:769] Sending connection context
0812 00:39:24.753727 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 00:39:24.754027 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1438,"mutex_wait_us":30,"thread_start_us":2286,"threads_started":1}
I20260812 00:39:24.754869 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.742843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.7.244.65:38301)
0812 00:39:24.745635 (+ 2792us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.745643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.745680 (+ 37us) server_negotiation.cc:407] Connection header received
0812 00:39:24.745775 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.745783 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.746026 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.746730 (+ 704us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:24.746927 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.747927 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.751166 (+ 3239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.752869 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.753045 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.753447 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.754274 (+ 827us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.754495 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2621,"thread_start_us":2478,"threads_started":1}
I20260812 00:39:24.754508 17436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.742704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.65:38301 (local address 127.0.0.1:45696)
0812 00:39:24.743681 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:24.743737 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:24.743937 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:24.746236 (+ 2299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:24.746240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:24.746266 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:24.746576 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.746585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.748093 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.748099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.750916 (+ 2817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.750933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.753594 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.753600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.753881 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.753916 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 00:39:24.754028 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 00:39:24.754149 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":76,"thread_start_us":393,"threads_started":1}
I20260812 00:39:24.755538 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.743632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:44121 (local address 127.0.0.1:38886)
0812 00:39:24.744991 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:24.745007 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:24.745101 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:24.749193 (+ 4092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:24.749212 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:24.749244 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:24.749905 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.749922 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.751417 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.751422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.752821 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.752841 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.754804 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.754810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.755007 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.755040 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 00:39:24.755102 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 00:39:24.755231 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1242,"thread_start_us":1200,"threads_started":1}
I20260812 00:39:24.755623 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.744036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.7.244.67:44581)
0812 00:39:24.744721 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.744730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.744768 (+ 38us) server_negotiation.cc:407] Connection header received
0812 00:39:24.744882 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.744891 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.745170 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.746235 (+ 1065us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:24.746562 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.748260 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.751504 (+ 3244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.752964 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.754640 (+ 1676us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.754961 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.755056 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.755242 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":946,"threads_started":1}
I20260812 00:39:24.756244 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.744655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.7.244.66:44121)
0812 00:39:24.748511 (+ 3856us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.748531 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.748563 (+ 32us) server_negotiation.cc:407] Connection header received
0812 00:39:24.748679 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.748688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.748957 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.749876 (+ 919us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:24.750210 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.751257 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.753103 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.754577 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.755420 (+ 843us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.755667 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.755762 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.755962 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3674,"thread_start_us":3543,"threads_started":1}
I20260812 00:39:24.759675 17003 tablet_service.cc:1467] Processing CreateTablet for tablet ff2002854f584b9396ad5d0a9cb1b5bf (DEFAULT_TABLE table=test-table [id=aae5803c865e457c8668bf16f7036222]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:24.759728 17263 tablet_service.cc:1467] Processing CreateTablet for tablet ff2002854f584b9396ad5d0a9cb1b5bf (DEFAULT_TABLE table=test-table [id=aae5803c865e457c8668bf16f7036222]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:24.760437 16726 tablet_service.cc:1467] Processing CreateTablet for tablet ff2002854f584b9396ad5d0a9cb1b5bf (DEFAULT_TABLE table=test-table [id=aae5803c865e457c8668bf16f7036222]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 00:39:24.761001 17003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2002854f584b9396ad5d0a9cb1b5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:24.761273 17263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2002854f584b9396ad5d0a9cb1b5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:24.761840 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff2002854f584b9396ad5d0a9cb1b5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:24.773721 17449 tablet_bootstrap.cc:492] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Bootstrap starting.
I20260812 00:39:24.776525 17451 tablet_bootstrap.cc:492] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Bootstrap starting.
I20260812 00:39:24.778044 17454 tablet_bootstrap.cc:492] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Bootstrap starting.
I20260812 00:39:24.780064 17449 tablet_bootstrap.cc:654] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:24.781005 17451 tablet_bootstrap.cc:654] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:24.781142 17454 tablet_bootstrap.cc:654] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:24.781548 17449 log.cc:824] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:24.782248 17454 log.cc:824] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:24.782362 17451 log.cc:824] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:24.785388 17451 tablet_bootstrap.cc:492] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: No bootstrap required, opened a new log
I20260812 00:39:24.785436 17449 tablet_bootstrap.cc:492] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: No bootstrap required, opened a new log
I20260812 00:39:24.785667 17454 tablet_bootstrap.cc:492] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: No bootstrap required, opened a new log
I20260812 00:39:24.785759 17449 ts_tablet_manager.cc:1397] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20260812 00:39:24.785756 17451 ts_tablet_manager.cc:1397] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 00:39:24.786937 17454 ts_tablet_manager.cc:1397] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 00:39:24.793852 17451 raft_consensus.cc:348] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.794327 17451 raft_consensus.cc:374] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:24.794452 17451 raft_consensus.cc:729] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 414ce8a89f384a9ba01551d14eb5ec93, State: Initialized, Role: FOLLOWER
I20260812 00:39:24.795140 17451 consensus_queue.cc:260] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.796401 17411 heartbeater.cc:503] Master 127.7.244.126:43621 was elected leader, sending a full tablet report...
I20260812 00:39:24.797224 17451 ts_tablet_manager.cc:1428] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260812 00:39:24.797035 17449 raft_consensus.cc:348] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.797713 17449 raft_consensus.cc:374] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:24.797825 17449 raft_consensus.cc:729] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f62f63c0c5c4d70bfcb4fa0d10d8328, State: Initialized, Role: FOLLOWER
I20260812 00:39:24.798007 17454 raft_consensus.cc:348] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.798816 17449 consensus_queue.cc:260] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.799137 17454 raft_consensus.cc:374] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:24.799310 17454 raft_consensus.cc:729] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9914fd29217d4e98bf184b19f6d6ab29, State: Initialized, Role: FOLLOWER
I20260812 00:39:24.800150 17454 consensus_queue.cc:260] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.802719 17141 heartbeater.cc:503] Master 127.7.244.126:43621 was elected leader, sending a full tablet report...
I20260812 00:39:24.802772 17454 ts_tablet_manager.cc:1428] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
I20260812 00:39:24.803555 17449 ts_tablet_manager.cc:1428] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
I20260812 00:39:24.854784 17464 raft_consensus.cc:482] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 00:39:24.855130 17464 raft_consensus.cc:504] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.857076 17464 leader_election.cc:288] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 414ce8a89f384a9ba01551d14eb5ec93 (127.7.244.67:44581), 3f62f63c0c5c4d70bfcb4fa0d10d8328 (127.7.244.66:44121)
I20260812 00:39:24.866374 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.857921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:39331 (local address 127.7.244.67:44581)
0812 00:39:24.858156 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.858162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.858787 (+ 625us) server_negotiation.cc:407] Connection header received
0812 00:39:24.859008 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.859013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.859074 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.859162 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:24.859792 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.860880 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.864011 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.865279 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.865497 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.865572 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.865983 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.866119 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:39:24.866405 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.857945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.244.65:47389 (local address 127.7.244.66:44121)
0812 00:39:24.858156 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.858162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.859133 (+ 971us) server_negotiation.cc:407] Connection header received
0812 00:39:24.859305 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.859311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.859383 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.859455 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 00:39:24.860073 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.861080 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.864001 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.865269 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.865497 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.865571 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.865983 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.866119 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:39:24.866528 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.857618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.66:44121 (local address 127.7.244.65:47389)
0812 00:39:24.859029 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:24.859043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:24.859169 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:24.859511 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:24.859515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:24.859541 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:24.859897 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.859907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.861217 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.861226 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.863719 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.863740 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.865544 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.865557 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.865766 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.865837 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 00:39:24.865930 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 00:39:24.866104 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1312,"mutex_wait_us":82,"thread_start_us":705,"threads_started":1}
I20260812 00:39:24.866554 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.857391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.67:44581 (local address 127.7.244.65:39331)
0812 00:39:24.858661 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0812 00:39:24.858678 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:39:24.858864 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:39:24.859211 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:39:24.859216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:39:24.859242 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 00:39:24.859598 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.859610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.861037 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.861041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.863723 (+ 2682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:39:24.863740 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.865578 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.865585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:39:24.865768 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.865833 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 00:39:24.865930 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 00:39:24.866104 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"mutex_wait_us":261,"thread_start_us":1338,"threads_started":1}
I20260812 00:39:24.867643 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2002854f584b9396ad5d0a9cb1b5bf" candidate_uuid: "9914fd29217d4e98bf184b19f6d6ab29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" is_pre_election: true
I20260812 00:39:24.867645 17318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2002854f584b9396ad5d0a9cb1b5bf" candidate_uuid: "9914fd29217d4e98bf184b19f6d6ab29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "414ce8a89f384a9ba01551d14eb5ec93" is_pre_election: true
I20260812 00:39:24.868137 17318 raft_consensus.cc:2454] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9914fd29217d4e98bf184b19f6d6ab29 in term 0.
I20260812 00:39:24.868137 17038 raft_consensus.cc:2454] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9914fd29217d4e98bf184b19f6d6ab29 in term 0.
I20260812 00:39:24.868796 16644 leader_election.cc:302] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f62f63c0c5c4d70bfcb4fa0d10d8328, 9914fd29217d4e98bf184b19f6d6ab29; no voters:
I20260812 00:39:24.869197 17464 raft_consensus.cc:2781] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 00:39:24.869390 17464 raft_consensus.cc:482] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 00:39:24.869501 17464 raft_consensus.cc:3037] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:24.871703 17464 raft_consensus.cc:504] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.872365 17464 leader_election.cc:288] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [CANDIDATE]: Term 1 election: Requested vote from peers 414ce8a89f384a9ba01551d14eb5ec93 (127.7.244.67:44581), 3f62f63c0c5c4d70bfcb4fa0d10d8328 (127.7.244.66:44121)
I20260812 00:39:24.872792 17318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2002854f584b9396ad5d0a9cb1b5bf" candidate_uuid: "9914fd29217d4e98bf184b19f6d6ab29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "414ce8a89f384a9ba01551d14eb5ec93"
I20260812 00:39:24.872972 17038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff2002854f584b9396ad5d0a9cb1b5bf" candidate_uuid: "9914fd29217d4e98bf184b19f6d6ab29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328"
I20260812 00:39:24.873031 17318 raft_consensus.cc:3037] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:24.873186 17038 raft_consensus.cc:3037] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:24.876362 17038 raft_consensus.cc:2454] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9914fd29217d4e98bf184b19f6d6ab29 in term 1.
I20260812 00:39:24.876362 17318 raft_consensus.cc:2454] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9914fd29217d4e98bf184b19f6d6ab29 in term 1.
I20260812 00:39:24.876865 16644 leader_election.cc:302] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3f62f63c0c5c4d70bfcb4fa0d10d8328, 414ce8a89f384a9ba01551d14eb5ec93, 9914fd29217d4e98bf184b19f6d6ab29; no voters:
I20260812 00:39:24.877254 17464 raft_consensus.cc:2781] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:24.877763 17464 raft_consensus.cc:686] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 1 LEADER]: Becoming Leader. State: Replica: 9914fd29217d4e98bf184b19f6d6ab29, State: Running, Role: LEADER
I20260812 00:39:24.878279 17464 consensus_queue.cc:237] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } }
I20260812 00:39:24.883663 16512 catalog_manager.cc:5495] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9914fd29217d4e98bf184b19f6d6ab29 (127.7.244.65). New cstate: current_term: 1 leader_uuid: "9914fd29217d4e98bf184b19f6d6ab29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9914fd29217d4e98bf184b19f6d6ab29" member_type: VOTER last_known_addr { host: "127.7.244.65" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 } health_report { overall_health: UNKNOWN } } }
I20260812 00:39:24.916023 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.910722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45710 (local address 127.7.244.65:38301)
0812 00:39:24.910925 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.910930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.910949 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:39:24.911107 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.911111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.911219 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.911352 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:39:24.911789 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.912508 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.913946 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.914357 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.914460 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.914907 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:39:24.915460 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:39:24.915575 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.915725 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.915802 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260812 00:39:24.924237 16860 tablet.cc:2369] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:24.925166 17038 raft_consensus.cc:1261] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 1 FOLLOWER]: Refusing update from remote peer 9914fd29217d4e98bf184b19f6d6ab29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:24.925267 17318 raft_consensus.cc:1261] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 1 FOLLOWER]: Refusing update from remote peer 9914fd29217d4e98bf184b19f6d6ab29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:39:24.926256 17471 consensus_queue.cc:1035] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f62f63c0c5c4d70bfcb4fa0d10d8328" member_type: VOTER last_known_addr { host: "127.7.244.66" port: 44121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:39:24.926611 17464 consensus_queue.cc:1035] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "414ce8a89f384a9ba01551d14eb5ec93" member_type: VOTER last_known_addr { host: "127.7.244.67" port: 44581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 00:39:24.931613 17415 tablet.cc:2369] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 00:39:24.938019 17147 tablet.cc:2369] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:39:24.941567 17482 mvcc.cc:205] Tried to move back new op lower bound from 7317484195522109440 to 7317484195345469440. Current Snapshot: MvccSnapshot[applied={T|T < 7317484195522109440}]
I20260812 00:39:24.950112 17483 mvcc.cc:205] Tried to move back new op lower bound from 7317484195522109440 to 7317484195345469440. Current Snapshot: MvccSnapshot[applied={T|T < 7317484195522109440}]
I20260812 00:39:24.960857 16512 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:24.961102 16512 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 00:39:24.965229 16512 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba6940a706d146d8a6e273043664f88d: Sending DeleteTablet for 3 replicas of tablet ff2002854f584b9396ad5d0a9cb1b5bf
I20260812 00:39:24.966791 16726 tablet_service.cc:1514] Processing DeleteTablet for tablet ff2002854f584b9396ad5d0a9cb1b5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:24 UTC) from {username='slave'} at 127.0.0.1:45696
I20260812 00:39:24.966791 17263 tablet_service.cc:1514] Processing DeleteTablet for tablet ff2002854f584b9396ad5d0a9cb1b5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:24 UTC) from {username='slave'} at 127.0.0.1:36440
I20260812 00:39:24.966786 17003 tablet_service.cc:1514] Processing DeleteTablet for tablet ff2002854f584b9396ad5d0a9cb1b5bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 00:39:24 UTC) from {username='slave'} at 127.0.0.1:38886
I20260812 00:39:24.967962 17513 tablet_replica.cc:330] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: stopping tablet replica
I20260812 00:39:24.968544 17513 raft_consensus.cc:2229] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 1 LEADER]: Raft consensus shutting down.
I20260812 00:39:24.969892 17511 tablet_replica.cc:330] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: stopping tablet replica
I20260812 00:39:24.970441 17511 raft_consensus.cc:2229] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:24.971122 17511 raft_consensus.cc:2258] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:24.971918 17513 raft_consensus.cc:2258] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:24.973549 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 16611
I20260812 00:39:24.974028 17511 ts_tablet_manager.cc:1905] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:24.978277 17512 tablet_replica.cc:330] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: stopping tablet replica
I20260812 00:39:24.978830 17512 raft_consensus.cc:2229] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 00:39:24.979442 17512 raft_consensus.cc:2258] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 00:39:24.981905 17513 ts_tablet_manager.cc:1905] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:24.982918 17511 ts_tablet_manager.cc:1918] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:24.983161 17511 log.cc:1196] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-1/wal/wals/ff2002854f584b9396ad5d0a9cb1b5bf
I20260812 00:39:24.983791 17511 ts_tablet_manager.cc:1939] T ff2002854f584b9396ad5d0a9cb1b5bf P 3f62f63c0c5c4d70bfcb4fa0d10d8328: Deleting consensus metadata
I20260812 00:39:24.986466 16469 catalog_manager.cc:4841] TS 3f62f63c0c5c4d70bfcb4fa0d10d8328 (127.7.244.66:44121): tablet ff2002854f584b9396ad5d0a9cb1b5bf (table test-table [id=aae5803c865e457c8668bf16f7036222]) successfully deleted
I20260812 00:39:24.989428 17512 ts_tablet_manager.cc:1905] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 00:39:24.990314 17513 ts_tablet_manager.cc:1918] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:24.990666 17513 log.cc:1196] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-0/wal/wals/ff2002854f584b9396ad5d0a9cb1b5bf
I20260812 00:39:24.991329 17513 ts_tablet_manager.cc:1939] T ff2002854f584b9396ad5d0a9cb1b5bf P 9914fd29217d4e98bf184b19f6d6ab29: Deleting consensus metadata
I20260812 00:39:24.993532 16463 catalog_manager.cc:4841] TS 9914fd29217d4e98bf184b19f6d6ab29 (127.7.244.65:38301): tablet ff2002854f584b9396ad5d0a9cb1b5bf (table test-table [id=aae5803c865e457c8668bf16f7036222]) successfully deleted
I20260812 00:39:24.995464 17437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:24.976718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.7.244.65:38301)
0812 00:39:24.976932 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:24.976939 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:24.986245 (+ 9306us) server_negotiation.cc:407] Connection header received
0812 00:39:24.986571 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:24.986581 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:24.986711 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:24.986794 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:24.988220 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.989493 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.991449 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:24.992084 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:24.992227 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:24.993932 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:24.993942 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:24.993957 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:24.994578 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:24.994654 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:24.994676 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:24.994745 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:24.994979 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 00:39:24.995091 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:39:24.996176 16851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:24.997582 17512 ts_tablet_manager.cc:1918] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 00:39:24.997818 17512 log.cc:1196] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Deleting WAL directory at /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786494824837501-8145-0/minicluster-data/ts-2/wal/wals/ff2002854f584b9396ad5d0a9cb1b5bf
I20260812 00:39:24.998431 17512 ts_tablet_manager.cc:1939] T ff2002854f584b9396ad5d0a9cb1b5bf P 414ce8a89f384a9ba01551d14eb5ec93: Deleting consensus metadata
I20260812 00:39:25.000623 16464 catalog_manager.cc:4841] TS 414ce8a89f384a9ba01551d14eb5ec93 (127.7.244.67:44581): tablet ff2002854f584b9396ad5d0a9cb1b5bf (table test-table [id=aae5803c865e457c8668bf16f7036222]) successfully deleted
I20260812 00:39:25.095687 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 16611
I20260812 00:39:25.125437 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 16866
I20260812 00:39:25.132356 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:25.126353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38894 (local address 127.7.244.66:44121)
0812 00:39:25.126530 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:25.126535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:25.126558 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:39:25.126669 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:25.126673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:25.126774 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:25.126858 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:25.127188 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:25.127956 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:25.129236 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:25.129734 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:25.129836 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:25.131026 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:25.131035 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:25.131052 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:25.131680 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:25.131733 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:25.131754 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:25.131844 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:25.131980 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 00:39:25.132081 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 00:39:25.133191 17127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:25.224390 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 16866
I20260812 00:39:25.255439 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 17155
I20260812 00:39:25.262632 17446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:25.256393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36444 (local address 127.7.244.67:44581)
0812 00:39:25.256626 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:25.256632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:25.256653 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:39:25.256848 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:25.256853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:25.256950 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:25.257040 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:25.257530 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:25.258266 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:25.259523 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:25.259927 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:25.260046 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:25.261170 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:25.261179 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:25.261195 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:25.261841 (+ 646us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:25.261896 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:25.261919 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:25.262026 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:25.262178 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 00:39:25.262316 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 00:39:25.263342 17402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:25.352842 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 17155
I20260812 00:39:25.384102 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 16434
I20260812 00:39:25.385512 16553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:25.495850 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 16434
2026-08-12T00:39:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3041 ms)
[----------] 6 tests from EncryptionPolicyTest (24234 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:25 dist-test-slave-gjw7 krb5kdc[17530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:25 dist-test-slave-gjw7 krb5kdc[17530](info): set up 2 sockets
Aug 12 00:39:25 dist-test-slave-gjw7 krb5kdc[17530](info): commencing operation
krb5kdc: starting...
W20260812 00:39:26.837610 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:27 dist-test-slave-gjw7 krb5kdc[17530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:27.165115 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37081
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:27.534996 17557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:27.535364 17557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:27.535499 17557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:27.543251 17557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:27.543406 17557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:27.543490 17557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:27.543572 17557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:27.543637 17557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:27.554083 17557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37081
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:27.555539 17557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:27.557066 17557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:27.569329 17569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:27.569371 17576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:27.569226 17572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:27.570446 17557 server_base.cc:1034] running on GCE node
I20260812 00:39:27.571293 17557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:27.572799 17557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:27.574041 17557 hybrid_clock.cc:648] HybridClock initialized: now 1786495167573993 us; error 33 us; skew 500 ppm
I20260812 00:39:27.574445 17557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:27 dist-test-slave-gjw7 krb5kdc[17530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:27.581873 17557 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:27.583822 17557 webserver.cc:460] Webserver started at http://127.0.0.1:35831/ using document root <none> and password file <none>
I20260812 00:39:27.584643 17557 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:27.584789 17557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:27.585151 17557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:27.588629 17557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "55a8d028d71347ca84f7d42cf6ce3826"
format_stamp: "Formatted at 2026-08-12 00:39:27 on dist-test-slave-gjw7"
server_key: "4f4dd6de071fa2f139901f45bc3eb4b5"
server_key_iv: "17a60d7f54dcf952b3c9b3c1df20e303"
server_key_version: "encryptionkey@0"
I20260812 00:39:27.589634 17557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "55a8d028d71347ca84f7d42cf6ce3826"
format_stamp: "Formatted at 2026-08-12 00:39:27 on dist-test-slave-gjw7"
server_key: "4f4dd6de071fa2f139901f45bc3eb4b5"
server_key_iv: "17a60d7f54dcf952b3c9b3c1df20e303"
server_key_version: "encryptionkey@0"
I20260812 00:39:27.598682 17557 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260812 00:39:27.606137 17589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:27.608323 17557 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.002s
I20260812 00:39:27.608565 17557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "55a8d028d71347ca84f7d42cf6ce3826"
format_stamp: "Formatted at 2026-08-12 00:39:27 on dist-test-slave-gjw7"
server_key: "4f4dd6de071fa2f139901f45bc3eb4b5"
server_key_iv: "17a60d7f54dcf952b3c9b3c1df20e303"
server_key_version: "encryptionkey@0"
I20260812 00:39:27.608863 17557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:27.641983 17557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:27.647410 17557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:27.647742 17557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:27.668233 17557 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37081
I20260812 00:39:27.668269 17689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37081 every 8 connection(s)
I20260812 00:39:27.669730 17557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:27.671963 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 17557
I20260812 00:39:27.672185 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:27.673053 8145 external_mini_cluster.cc:1428] Setting key 6567fcf42d3588db13ba356f96149e9f
I20260812 00:39:27.674448 17691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:27.687409 17691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826: Bootstrap starting.
Aug 12 00:39:27 dist-test-slave-gjw7 krb5kdc[17530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 00:39:27.691376 17691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:27.692617 17691 log.cc:824] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:27.696524 17691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826: No bootstrap required, opened a new log
I20260812 00:39:27.697806 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:27.675206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34977 (local address 127.0.0.1:37081)
0812 00:39:27.676065 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:27.676078 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:27.676122 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:39:27.677922 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:27.677949 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:27.678429 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:27.678846 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:27.680495 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:27.681934 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:27.683290 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:27.684052 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:27.684255 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:27.689062 (+ 4807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:27.689072 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:27.689094 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:27.689145 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:27.692482 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:27.693746 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:27.693756 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:27.693766 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:27.694023 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:27.694666 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:27.694670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:27.694673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:27.695308 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:27.695557 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:27.696273 (+ 716us) server_negotiation.cc:299] Negotiation successful
0812 00:39:27.696598 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":374,"threads_started":1}
I20260812 00:39:27.702407 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:27.706102 17691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a8d028d71347ca84f7d42cf6ce3826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37081 } }
I20260812 00:39:27.706667 17691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:27.706810 17691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55a8d028d71347ca84f7d42cf6ce3826, State: Initialized, Role: FOLLOWER
I20260812 00:39:27.707580 17691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a8d028d71347ca84f7d42cf6ce3826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37081 } }
I20260812 00:39:27.707913 17691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:27.708069 17691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:27.708246 17691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:27.710958 17691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a8d028d71347ca84f7d42cf6ce3826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37081 } }
I20260812 00:39:27.711656 17691 leader_election.cc:302] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55a8d028d71347ca84f7d42cf6ce3826; no voters:
I20260812 00:39:27.712424 17691 leader_election.cc:288] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:27.712597 17720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:27.714694 17691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:27.719621 17720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [term 1 LEADER]: Becoming Leader. State: Replica: 55a8d028d71347ca84f7d42cf6ce3826, State: Running, Role: LEADER
I20260812 00:39:27.720346 17720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a8d028d71347ca84f7d42cf6ce3826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37081 } }
I20260812 00:39:27.723636 17722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55a8d028d71347ca84f7d42cf6ce3826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a8d028d71347ca84f7d42cf6ce3826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37081 } } }
I20260812 00:39:27.724344 17722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:27.724196 17724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a8d028d71347ca84f7d42cf6ce3826. Latest consensus state: current_term: 1 leader_uuid: "55a8d028d71347ca84f7d42cf6ce3826" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a8d028d71347ca84f7d42cf6ce3826" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37081 } } }
I20260812 00:39:27.724481 17724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:27.725345 17743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:27.728823 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:27.708705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56343 (local address 127.0.0.1:37081)
0812 00:39:27.708899 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:27.708905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:27.709222 (+ 317us) server_negotiation.cc:407] Connection header received
0812 00:39:27.709389 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:27.709394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:27.709478 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:27.709601 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:27.711062 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:27.712202 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:27.714323 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:27.715039 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:27.715177 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:27.719194 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:27.719203 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:27.719208 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:27.719278 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:27.723774 (+ 4496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:27.725142 (+ 1368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:27.725149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:27.725154 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:27.725454 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:27.726159 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:27.726165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:27.726168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:27.726621 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:27.728017 (+ 1396us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:27.728210 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 00:39:27.728432 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:39:27.737267 17743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:27.748993 17743 catalog_manager.cc:1347] Generated new cluster ID: fc0bc5660fd645ae9296d7e4e708fc83
I20260812 00:39:27.749146 17743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:27.764710 17743 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:27.766580 17743 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:27.774605 17743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55a8d028d71347ca84f7d42cf6ce3826: Generated new TSK 0
I20260812 00:39:27.775874 17743 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:27.789654 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 17557
I20260812 00:39:27.791266 17684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:27.890673 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 17557
W20260812 00:39:27.919530 17525 connection.cc:535] client connection to 127.0.0.1:37081 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37081): Connection reset by peer (error 104)
2026-08-12T00:39:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:28 dist-test-slave-gjw7 krb5kdc[17769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:28 dist-test-slave-gjw7 krb5kdc[17769](info): set up 2 sockets
Aug 12 00:39:28 dist-test-slave-gjw7 krb5kdc[17769](info): commencing operation
krb5kdc: starting...
W20260812 00:39:29.228570 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:29 dist-test-slave-gjw7 krb5kdc[17769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:29.554198 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41067
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:29.929699 17802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:29.930086 17802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:29.930261 17802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:29.938032 17802 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:29.938181 17802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:29.938272 17802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:29.938376 17802 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:29.938444 17802 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:29.949258 17802 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:29.950845 17802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:29.952312 17802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:29.965566 17817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:29.965744 17816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:29.965577 17819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:29.966471 17802 server_base.cc:1034] running on GCE node
I20260812 00:39:29.967314 17802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:29.968837 17802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:29.970096 17802 hybrid_clock.cc:648] HybridClock initialized: now 1786495169970043 us; error 50 us; skew 500 ppm
I20260812 00:39:29.970531 17802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:29 dist-test-slave-gjw7 krb5kdc[17769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:29.978857 17802 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:29.980707 17802 webserver.cc:460] Webserver started at http://127.0.0.1:45373/ using document root <none> and password file <none>
I20260812 00:39:29.981570 17802 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:29.981721 17802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:29.982080 17802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:29.985607 17802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "a865755bd891478d8ed2da4909a224cc"
format_stamp: "Formatted at 2026-08-12 00:39:29 on dist-test-slave-gjw7"
server_key: "bd7153e692280c94c6719f622a2f12ae"
server_key_iv: "d4ec36282d4a483735ed8a786e7dc9c0"
server_key_version: "encryptionkey@0"
I20260812 00:39:29.986558 17802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "a865755bd891478d8ed2da4909a224cc"
format_stamp: "Formatted at 2026-08-12 00:39:29 on dist-test-slave-gjw7"
server_key: "bd7153e692280c94c6719f622a2f12ae"
server_key_iv: "d4ec36282d4a483735ed8a786e7dc9c0"
server_key_version: "encryptionkey@0"
I20260812 00:39:29.992765 17802 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:39:29.996973 17828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:29.998667 17802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:39:29.998875 17802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "a865755bd891478d8ed2da4909a224cc"
format_stamp: "Formatted at 2026-08-12 00:39:29 on dist-test-slave-gjw7"
server_key: "bd7153e692280c94c6719f622a2f12ae"
server_key_iv: "d4ec36282d4a483735ed8a786e7dc9c0"
server_key_version: "encryptionkey@0"
I20260812 00:39:29.999101 17802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:30.015820 17802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:30.021648 17802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:30.021988 17802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:30.043884 17802 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41067
I20260812 00:39:30.043922 17939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41067 every 8 connection(s)
I20260812 00:39:30.045490 17802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:30.047252 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 17802
I20260812 00:39:30.047446 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:30.048012 8145 external_mini_cluster.cc:1428] Setting key 975b79ccb80226beec5bb54800053884
I20260812 00:39:30.050896 17943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:30.063433 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:30.050000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47753 (local address 127.0.0.1:41067)
0812 00:39:30.050870 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:30.050886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:30.050940 (+ 54us) server_negotiation.cc:407] Connection header received
0812 00:39:30.052692 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:30.052724 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:30.053225 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:30.053639 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:30.054363 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:30.055841 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:30.058058 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:30.058818 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:30.058997 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:30.060567 (+ 1570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:30.060577 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:30.060596 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:30.061247 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:30.061386 (+ 139us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:30.061423 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:30.061586 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:30.061964 (+ 378us) server_negotiation.cc:299] Negotiation successful
0812 00:39:30.062205 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":417,"threads_started":1}
I20260812 00:39:30.065500 17943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc: Bootstrap starting.
I20260812 00:39:30.068109 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:30.069813 17943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:30.071513 17943 log.cc:824] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:30.075634 17943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc: No bootstrap required, opened a new log
I20260812 00:39:30.081275 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:30.074209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57181 (local address 127.0.0.1:41067)
0812 00:39:30.074426 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:30.074434 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:30.074860 (+ 426us) server_negotiation.cc:407] Connection header received
0812 00:39:30.075056 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:30.075063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:30.075156 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:30.075295 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:30.075827 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:30.077160 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:30.078626 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:30.079129 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:30.079259 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:30.080485 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:30.080491 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:30.080494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:30.080516 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:30.080566 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:30.080583 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:30.080697 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:30.080910 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 00:39:30.080972 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:39:30.085784 17943 raft_consensus.cc:348] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a865755bd891478d8ed2da4909a224cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } }
I20260812 00:39:30.086325 17943 raft_consensus.cc:374] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:30.086442 17943 raft_consensus.cc:729] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a865755bd891478d8ed2da4909a224cc, State: Initialized, Role: FOLLOWER
I20260812 00:39:30.087141 17943 consensus_queue.cc:260] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a865755bd891478d8ed2da4909a224cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } }
I20260812 00:39:30.087455 17943 raft_consensus.cc:388] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:30.087623 17943 raft_consensus.cc:482] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:30.087819 17943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:30.090714 17943 raft_consensus.cc:504] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a865755bd891478d8ed2da4909a224cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } }
I20260812 00:39:30.091391 17943 leader_election.cc:302] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a865755bd891478d8ed2da4909a224cc; no voters:
I20260812 00:39:30.092105 17943 leader_election.cc:288] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:30.092234 17961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:30.093123 17961 raft_consensus.cc:686] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [term 1 LEADER]: Becoming Leader. State: Replica: a865755bd891478d8ed2da4909a224cc, State: Running, Role: LEADER
I20260812 00:39:30.093802 17961 consensus_queue.cc:237] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a865755bd891478d8ed2da4909a224cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } }
I20260812 00:39:30.094732 17943 sys_catalog.cc:564] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:30.098038 17962 sys_catalog.cc:455] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a865755bd891478d8ed2da4909a224cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a865755bd891478d8ed2da4909a224cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } } }
I20260812 00:39:30.098127 17964 sys_catalog.cc:455] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a865755bd891478d8ed2da4909a224cc. Latest consensus state: current_term: 1 leader_uuid: "a865755bd891478d8ed2da4909a224cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a865755bd891478d8ed2da4909a224cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41067 } } }
I20260812 00:39:30.098707 17964 sys_catalog.cc:458] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:30.098706 17962 sys_catalog.cc:458] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:30.109566 17976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:30.114408 17976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:30.123308 17976 catalog_manager.cc:1347] Generated new cluster ID: 5c2f71b22ac34d9983249c246c88ee27
I20260812 00:39:30.123452 17976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:30.164361 17976 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:30.165863 17976 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:30.173663 17976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a865755bd891478d8ed2da4909a224cc: Generated new TSK 0
I20260812 00:39:30.174602 17976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:30.186265 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 17802
W20260812 00:39:30.187505 17935 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47753
W20260812 00:39:30.188510 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 17802: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:30.188709 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 17802
W20260812 00:39:30.216159 17766 connection.cc:535] client connection to 127.0.0.1:41067 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41067): Connection reset by peer (error 104)
2026-08-12T00:39:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2296 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:30 dist-test-slave-gjw7 krb5kdc[18001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:30 dist-test-slave-gjw7 krb5kdc[18001](info): set up 2 sockets
Aug 12 00:39:30 dist-test-slave-gjw7 krb5kdc[18001](info): commencing operation
krb5kdc: starting...
W20260812 00:39:31.528661 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:31 dist-test-slave-gjw7 krb5kdc[18001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:31.857513 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44619
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44619
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:32.228026 18027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:32.228399 18027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:32.228529 18027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:32.236232 18027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:32.236377 18027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:32.236428 18027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:32.236531 18027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:32.236583 18027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:32.247087 18027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44619
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:32.248585 18027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:32.250228 18027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:32.262888 18039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:32.262925 18041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:32.262953 18037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:32.263617 18027 server_base.cc:1034] running on GCE node
I20260812 00:39:32.264528 18027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:32.266182 18027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:32.267428 18027 hybrid_clock.cc:648] HybridClock initialized: now 1786495172267376 us; error 40 us; skew 500 ppm
I20260812 00:39:32.267814 18027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:32 dist-test-slave-gjw7 krb5kdc[18001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:32.271453 18027 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:32.273197 18027 webserver.cc:460] Webserver started at http://127.0.0.1:45695/ using document root <none> and password file <none>
I20260812 00:39:32.273967 18027 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:32.274092 18027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:32.274428 18027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:32.277871 18027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "f7889c2e18cc4ea7a0bf7101f7af069d"
format_stamp: "Formatted at 2026-08-12 00:39:32 on dist-test-slave-gjw7"
server_key: "a2b6ab07a9f5d1283bd76b3650b42df8"
server_key_iv: "8824cde7b2c66d44235b0d4a916ccbd0"
server_key_version: "encryptionkey@0"
I20260812 00:39:32.278782 18027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "f7889c2e18cc4ea7a0bf7101f7af069d"
format_stamp: "Formatted at 2026-08-12 00:39:32 on dist-test-slave-gjw7"
server_key: "a2b6ab07a9f5d1283bd76b3650b42df8"
server_key_iv: "8824cde7b2c66d44235b0d4a916ccbd0"
server_key_version: "encryptionkey@0"
I20260812 00:39:32.284726 18027 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:39:32.288900 18049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:32.290482 18027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:39:32.290683 18027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "f7889c2e18cc4ea7a0bf7101f7af069d"
format_stamp: "Formatted at 2026-08-12 00:39:32 on dist-test-slave-gjw7"
server_key: "a2b6ab07a9f5d1283bd76b3650b42df8"
server_key_iv: "8824cde7b2c66d44235b0d4a916ccbd0"
server_key_version: "encryptionkey@0"
I20260812 00:39:32.290912 18027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:32.297221 18027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:32.301851 18027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:32.302174 18027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:32.321246 18027 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44619
I20260812 00:39:32.321327 18164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44619 every 8 connection(s)
I20260812 00:39:32.322826 18027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:32.327097 18167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:32.331054 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 18027
I20260812 00:39:32.331295 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:32.331753 8145 external_mini_cluster.cc:1428] Setting key 889c812d83dffb0211fd411c7a9e07d2
I20260812 00:39:32.340152 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:32.333661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47909 (local address 127.0.0.1:44619)
0812 00:39:32.334533 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:32.334545 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:32.334593 (+ 48us) server_negotiation.cc:407] Connection header received
0812 00:39:32.336336 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:32.336363 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:32.336799 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:32.337182 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:32.338366 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:32.338374 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:32.338389 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:32.338404 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:32.338475 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:32.338501 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:32.338719 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:32.338864 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 00:39:32.339113 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":382,"threads_started":1}
I20260812 00:39:32.340720 18167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d: Bootstrap starting.
I20260812 00:39:32.344254 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:32.344374 18167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:32.345504 18167 log.cc:824] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:32.349017 18167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d: No bootstrap required, opened a new log
I20260812 00:39:32.351637 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:32.349026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50443 (local address 127.0.0.1:44619)
0812 00:39:32.349221 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:32.349227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:32.349645 (+ 418us) server_negotiation.cc:407] Connection header received
0812 00:39:32.349793 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:32.349800 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:32.349867 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:32.349992 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:32.350873 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:32.350879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:32.350884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:32.350907 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:32.350963 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:32.350988 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:32.351116 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:32.351271 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 00:39:32.351337 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 00:39:32.357661 18167 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7889c2e18cc4ea7a0bf7101f7af069d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } }
I20260812 00:39:32.358131 18167 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:32.358217 18167 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7889c2e18cc4ea7a0bf7101f7af069d, State: Initialized, Role: FOLLOWER
I20260812 00:39:32.358886 18167 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7889c2e18cc4ea7a0bf7101f7af069d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } }
I20260812 00:39:32.359189 18167 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:32.359340 18167 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:32.359524 18167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:32.361979 18167 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7889c2e18cc4ea7a0bf7101f7af069d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } }
I20260812 00:39:32.362555 18167 leader_election.cc:302] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7889c2e18cc4ea7a0bf7101f7af069d; no voters:
I20260812 00:39:32.363263 18167 leader_election.cc:288] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:32.363446 18187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:32.364192 18187 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [term 1 LEADER]: Becoming Leader. State: Replica: f7889c2e18cc4ea7a0bf7101f7af069d, State: Running, Role: LEADER
I20260812 00:39:32.365052 18187 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7889c2e18cc4ea7a0bf7101f7af069d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } }
I20260812 00:39:32.365825 18167 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:32.368284 18194 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7889c2e18cc4ea7a0bf7101f7af069d. Latest consensus state: current_term: 1 leader_uuid: "f7889c2e18cc4ea7a0bf7101f7af069d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7889c2e18cc4ea7a0bf7101f7af069d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } } }
I20260812 00:39:32.368273 18190 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7889c2e18cc4ea7a0bf7101f7af069d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7889c2e18cc4ea7a0bf7101f7af069d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44619 } } }
I20260812 00:39:32.368839 18194 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:32.368896 18190 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:32.370476 18210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:32.377199 18210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:32.388105 18210 catalog_manager.cc:1347] Generated new cluster ID: e5a6464551904c79a5a9a5ac6406a41b
I20260812 00:39:32.388252 18210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:32.422600 18210 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:32.423156 18210 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:32.438117 18210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7889c2e18cc4ea7a0bf7101f7af069d: Generated new TSK 0
I20260812 00:39:32.439248 18210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:32.455204 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18027
W20260812 00:39:32.456645 18151 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47909
W20260812 00:39:32.457587 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18027: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:32.457767 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 18027
2026-08-12T00:39:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2266 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:32 dist-test-slave-gjw7 krb5kdc[18233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:32 dist-test-slave-gjw7 krb5kdc[18233](info): set up 2 sockets
Aug 12 00:39:32 dist-test-slave-gjw7 krb5kdc[18233](info): commencing operation
krb5kdc: starting...
W20260812 00:39:33.789629 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:34 dist-test-slave-gjw7 krb5kdc[18233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:34.122077 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:34.488602 18266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:34.488955 18266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:34.489096 18266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:34.496918 18266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:34.497074 18266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:34.497160 18266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:34.497257 18266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:34.497361 18266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:34.508256 18266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:34.509802 18266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:34.511291 18266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:34.525377 18277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:34.525421 18280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:34.525344 18284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:34.526237 18266 server_base.cc:1034] running on GCE node
I20260812 00:39:34.527386 18266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:34.529112 18266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:34.530367 18266 hybrid_clock.cc:648] HybridClock initialized: now 1786495174530307 us; error 94 us; skew 500 ppm
I20260812 00:39:34.530789 18266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:34 dist-test-slave-gjw7 krb5kdc[18233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:34.534452 18266 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:34.536337 18266 webserver.cc:460] Webserver started at http://127.0.0.1:33925/ using document root <none> and password file <none>
I20260812 00:39:34.537176 18266 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:34.537352 18266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:34.537706 18266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:34.541064 18266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "7daef44ea6f44dc68806968a4c4cd1bc"
format_stamp: "Formatted at 2026-08-12 00:39:34 on dist-test-slave-gjw7"
server_key: "b65f2f7b353527edfc1ed610e38000bc"
server_key_iv: "18f2e6545243d9d5717b0da7e977cafc"
server_key_version: "encryptionkey@0"
I20260812 00:39:34.542032 18266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "7daef44ea6f44dc68806968a4c4cd1bc"
format_stamp: "Formatted at 2026-08-12 00:39:34 on dist-test-slave-gjw7"
server_key: "b65f2f7b353527edfc1ed610e38000bc"
server_key_iv: "18f2e6545243d9d5717b0da7e977cafc"
server_key_version: "encryptionkey@0"
I20260812 00:39:34.548483 18266 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:39:34.552875 18298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:34.554620 18266 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 00:39:34.554862 18266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "7daef44ea6f44dc68806968a4c4cd1bc"
format_stamp: "Formatted at 2026-08-12 00:39:34 on dist-test-slave-gjw7"
server_key: "b65f2f7b353527edfc1ed610e38000bc"
server_key_iv: "18f2e6545243d9d5717b0da7e977cafc"
server_key_version: "encryptionkey@0"
I20260812 00:39:34.555097 18266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:34.573693 18266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:34.578518 18266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:34.578833 18266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:34.598623 18266 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35049
I20260812 00:39:34.598652 18401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35049 every 8 connection(s)
I20260812 00:39:34.600181 18266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:34.604404 18402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:34.605607 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 18266
I20260812 00:39:34.605789 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:34.606261 8145 external_mini_cluster.cc:1428] Setting key 9c7505511f1f0dc7d634fc3ac9aa2a96
I20260812 00:39:34.618371 18402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc: Bootstrap starting.
Aug 12 00:39:34 dist-test-slave-gjw7 krb5kdc[18233](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 00:39:34.622174 18402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:34.623683 18402 log.cc:824] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:34.628000 18402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc: No bootstrap required, opened a new log
I20260812 00:39:34.633370 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:34.608325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48739 (local address 127.0.0.1:35049)
0812 00:39:34.609174 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:34.609188 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:34.609249 (+ 61us) server_negotiation.cc:407] Connection header received
0812 00:39:34.611276 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:34.611309 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:34.611818 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:34.612246 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:34.613940 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:34.615320 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:34.616733 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:34.617549 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:34.617738 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:34.622514 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:34.622528 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:34.622560 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:34.622626 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:34.626993 (+ 4367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:34.628407 (+ 1414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:34.628419 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:34.628431 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:34.628767 (+ 336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:34.629348 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:34.629355 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:34.629358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:34.630258 (+ 900us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:34.630619 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:34.631098 (+ 479us) server_negotiation.cc:299] Negotiation successful
0812 00:39:34.631685 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":356,"threads_started":1}
I20260812 00:39:34.638062 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:34.639215 18402 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daef44ea6f44dc68806968a4c4cd1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260812 00:39:34.639653 18402 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:34.639770 18402 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7daef44ea6f44dc68806968a4c4cd1bc, State: Initialized, Role: FOLLOWER
I20260812 00:39:34.640450 18402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daef44ea6f44dc68806968a4c4cd1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260812 00:39:34.640730 18402 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:34.640885 18402 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:34.641070 18402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:34.644187 18402 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daef44ea6f44dc68806968a4c4cd1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260812 00:39:34.644804 18402 leader_election.cc:302] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7daef44ea6f44dc68806968a4c4cd1bc; no voters:
I20260812 00:39:34.645555 18402 leader_election.cc:288] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:34.645712 18425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:34.648476 18402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:34.652853 18425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [term 1 LEADER]: Becoming Leader. State: Replica: 7daef44ea6f44dc68806968a4c4cd1bc, State: Running, Role: LEADER
I20260812 00:39:34.653826 18425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daef44ea6f44dc68806968a4c4cd1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260812 00:39:34.658685 18426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7daef44ea6f44dc68806968a4c4cd1bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daef44ea6f44dc68806968a4c4cd1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } } }
I20260812 00:39:34.659195 18426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:34.658676 18428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7daef44ea6f44dc68806968a4c4cd1bc. Latest consensus state: current_term: 1 leader_uuid: "7daef44ea6f44dc68806968a4c4cd1bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7daef44ea6f44dc68806968a4c4cd1bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } } }
I20260812 00:39:34.659507 18428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:34.659986 18406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:34.643280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38573 (local address 127.0.0.1:35049)
0812 00:39:34.643451 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:34.643457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:34.643949 (+ 492us) server_negotiation.cc:407] Connection header received
0812 00:39:34.644134 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:34.644139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:34.644220 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:34.644337 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:34.646080 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:34.647150 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:34.648550 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:34.649084 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:34.649204 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:34.651868 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:34.651874 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:34.651877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:34.651917 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:34.655183 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:34.656290 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:34.656297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:34.656301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:34.656456 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:34.656961 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:34.656966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:34.656969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:34.657491 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:34.657791 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:34.659276 (+ 1485us) server_negotiation.cc:299] Negotiation successful
0812 00:39:34.659608 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 00:39:34.677999 18463 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:39:34.678191 18463 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:39:34.678346 18455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:34.680154 18455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:34.689368 18455 catalog_manager.cc:1347] Generated new cluster ID: 4ddde9e280ec4fb993a14883f59c47fe
I20260812 00:39:34.689504 18455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:34.713421 18455 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:34.715466 18455 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:34.724848 18455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7daef44ea6f44dc68806968a4c4cd1bc: Generated new TSK 0
I20260812 00:39:34.726137 18455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:34.743228 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18266
I20260812 00:39:34.744688 18388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:34.841655 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 18266
2026-08-12T00:39:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:34 dist-test-slave-gjw7 krb5kdc[18472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:34 dist-test-slave-gjw7 krb5kdc[18472](info): set up 2 sockets
Aug 12 00:39:34 dist-test-slave-gjw7 krb5kdc[18472](info): commencing operation
krb5kdc: starting...
W20260812 00:39:36.199442 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:36 dist-test-slave-gjw7 krb5kdc[18472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:36.530027 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37893
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:36.905597 18498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:36.905961 18498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:36.906119 18498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:36.913986 18498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:36.914135 18498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:36.914222 18498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:36.914328 18498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:36.914394 18498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:36.925222 18498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:36.926783 18498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:36.928242 18498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:36.941239 18504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:36.941166 18512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:36.941203 18506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:36.942279 18498 server_base.cc:1034] running on GCE node
I20260812 00:39:36.943243 18498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:36.944932 18498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:36.946200 18498 hybrid_clock.cc:648] HybridClock initialized: now 1786495176946141 us; error 37 us; skew 500 ppm
I20260812 00:39:36.946599 18498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:36 dist-test-slave-gjw7 krb5kdc[18472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495176, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:36.954453 18498 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:36.956257 18498 webserver.cc:460] Webserver started at http://127.0.0.1:35097/ using document root <none> and password file <none>
I20260812 00:39:36.957080 18498 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:36.957211 18498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:36.957582 18498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:36.961089 18498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "f91ef5bb66804570851c8cd0b75d5d22"
format_stamp: "Formatted at 2026-08-12 00:39:36 on dist-test-slave-gjw7"
server_key: "21cc19bb977fa31156323d2110353b3c"
server_key_iv: "4f5ffa8dacd6cc040b4100a46af7f4fa"
server_key_version: "encryptionkey@0"
I20260812 00:39:36.962107 18498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "f91ef5bb66804570851c8cd0b75d5d22"
format_stamp: "Formatted at 2026-08-12 00:39:36 on dist-test-slave-gjw7"
server_key: "21cc19bb977fa31156323d2110353b3c"
server_key_iv: "4f5ffa8dacd6cc040b4100a46af7f4fa"
server_key_version: "encryptionkey@0"
I20260812 00:39:36.968195 18498 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260812 00:39:36.972142 18524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:36.973765 18498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:39:36.973960 18498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "f91ef5bb66804570851c8cd0b75d5d22"
format_stamp: "Formatted at 2026-08-12 00:39:36 on dist-test-slave-gjw7"
server_key: "21cc19bb977fa31156323d2110353b3c"
server_key_iv: "4f5ffa8dacd6cc040b4100a46af7f4fa"
server_key_version: "encryptionkey@0"
I20260812 00:39:36.974184 18498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:36.994932 18498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:36.999698 18498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:37.000000 18498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:37.019727 18498 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37893
I20260812 00:39:37.019759 18628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37893 every 8 connection(s)
I20260812 00:39:37.021335 18498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:37.024179 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 18498
I20260812 00:39:37.024402 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:37.024957 8145 external_mini_cluster.cc:1428] Setting key 0be63391bd55893b7c18170b3a1f1116
I20260812 00:39:37.026355 18629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:37.042048 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:37.027017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56501 (local address 127.0.0.1:37893)
0812 00:39:37.028027 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:37.028044 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:37.028106 (+ 62us) server_negotiation.cc:407] Connection header received
0812 00:39:37.030253 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:37.030289 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:37.030958 (+ 669us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:37.031520 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:37.032175 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:37.034396 (+ 2221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:37.035962 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:37.037075 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:37.037321 (+ 246us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:37.038662 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:37.038674 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:37.038698 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:37.039559 (+ 861us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:37.039677 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:37.039725 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:37.039869 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:37.040173 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 00:39:37.040540 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":455,"threads_started":1}
I20260812 00:39:37.042939 18629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22: Bootstrap starting.
I20260812 00:39:37.046500 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:37.046651 18629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:37.048180 18629 log.cc:824] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:37.051788 18629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22: No bootstrap required, opened a new log
I20260812 00:39:37.059152 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:37.051677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38993 (local address 127.0.0.1:37893)
0812 00:39:37.051885 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:37.051892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:37.052389 (+ 497us) server_negotiation.cc:407] Connection header received
0812 00:39:37.052617 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:37.052621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:37.052689 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:37.052771 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:37.053581 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:37.054376 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:37.056373 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:37.056869 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:37.056985 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:37.058344 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:37.058350 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:37.058356 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:37.058379 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:37.058434 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:37.058453 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:37.058540 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:37.058697 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 00:39:37.058807 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 00:39:37.061934 18629 raft_consensus.cc:348] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91ef5bb66804570851c8cd0b75d5d22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260812 00:39:37.062391 18629 raft_consensus.cc:374] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:37.062500 18629 raft_consensus.cc:729] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f91ef5bb66804570851c8cd0b75d5d22, State: Initialized, Role: FOLLOWER
I20260812 00:39:37.063150 18629 consensus_queue.cc:260] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91ef5bb66804570851c8cd0b75d5d22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260812 00:39:37.063407 18629 raft_consensus.cc:388] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:37.063555 18629 raft_consensus.cc:482] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:37.063722 18629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:37.066061 18629 raft_consensus.cc:504] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91ef5bb66804570851c8cd0b75d5d22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260812 00:39:37.066622 18629 leader_election.cc:302] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f91ef5bb66804570851c8cd0b75d5d22; no voters:
I20260812 00:39:37.067269 18629 leader_election.cc:288] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:37.067471 18643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:37.068591 18643 raft_consensus.cc:686] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [term 1 LEADER]: Becoming Leader. State: Replica: f91ef5bb66804570851c8cd0b75d5d22, State: Running, Role: LEADER
I20260812 00:39:37.069165 18643 consensus_queue.cc:237] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91ef5bb66804570851c8cd0b75d5d22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } }
I20260812 00:39:37.069648 18629 sys_catalog.cc:564] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:37.072016 18646 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f91ef5bb66804570851c8cd0b75d5d22. Latest consensus state: current_term: 1 leader_uuid: "f91ef5bb66804570851c8cd0b75d5d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91ef5bb66804570851c8cd0b75d5d22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } } }
I20260812 00:39:37.072497 18646 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:37.072322 18644 sys_catalog.cc:455] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f91ef5bb66804570851c8cd0b75d5d22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f91ef5bb66804570851c8cd0b75d5d22" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37893 } } }
I20260812 00:39:37.072721 18644 sys_catalog.cc:458] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:37.074607 18662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:37.080456 18662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:37.091987 18662 catalog_manager.cc:1347] Generated new cluster ID: dab3bef144914293808204917b506a84
I20260812 00:39:37.092120 18662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:37.121192 18662 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:37.122709 18662 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:37.133266 18662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f91ef5bb66804570851c8cd0b75d5d22: Generated new TSK 0
I20260812 00:39:37.134557 18662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:37.150874 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18498
W20260812 00:39:37.152451 18620 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56501
W20260812 00:39:37.153184 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18498: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:37.153364 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 18498
2026-08-12T00:39:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:37 dist-test-slave-gjw7 krb5kdc[18683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:37 dist-test-slave-gjw7 krb5kdc[18683](info): set up 2 sockets
Aug 12 00:39:37 dist-test-slave-gjw7 krb5kdc[18683](info): commencing operation
krb5kdc: starting...
W20260812 00:39:38.510653 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.006s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:38 dist-test-slave-gjw7 krb5kdc[18683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:38.841014 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38969
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:39.204414 18700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:39.204747 18700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:39.204851 18700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:39.212299 18700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:39.212435 18700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:39.212482 18700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:39.212544 18700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:39.212589 18700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:39.222987 18700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:39.224349 18700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:39.225755 18700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:39.238350 18709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:39.238301 18718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:39.238274 18714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:39.239177 18700 server_base.cc:1034] running on GCE node
I20260812 00:39:39.240051 18700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:39.241673 18700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:39.242923 18700 hybrid_clock.cc:648] HybridClock initialized: now 1786495179242869 us; error 51 us; skew 500 ppm
I20260812 00:39:39.243304 18700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:39 dist-test-slave-gjw7 krb5kdc[18683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:39.246767 18700 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:39.248471 18700 webserver.cc:460] Webserver started at http://127.0.0.1:43309/ using document root <none> and password file <none>
I20260812 00:39:39.249248 18700 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:39.249419 18700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:39.249738 18700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:39.252986 18700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "d1e9adb280a8414bb9704ca8214e4582"
format_stamp: "Formatted at 2026-08-12 00:39:39 on dist-test-slave-gjw7"
server_key: "9988da6a5259050418d8fbacad83e21a"
server_key_iv: "78fbc81b511cbc50cadb6c3052a2b7b9"
server_key_version: "encryptionkey@0"
I20260812 00:39:39.253870 18700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "d1e9adb280a8414bb9704ca8214e4582"
format_stamp: "Formatted at 2026-08-12 00:39:39 on dist-test-slave-gjw7"
server_key: "9988da6a5259050418d8fbacad83e21a"
server_key_iv: "78fbc81b511cbc50cadb6c3052a2b7b9"
server_key_version: "encryptionkey@0"
I20260812 00:39:39.259718 18700 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:39:39.263752 18737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:39.265400 18700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 00:39:39.265610 18700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "d1e9adb280a8414bb9704ca8214e4582"
format_stamp: "Formatted at 2026-08-12 00:39:39 on dist-test-slave-gjw7"
server_key: "9988da6a5259050418d8fbacad83e21a"
server_key_iv: "78fbc81b511cbc50cadb6c3052a2b7b9"
server_key_version: "encryptionkey@0"
I20260812 00:39:39.265847 18700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:39.272185 18700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:39.276955 18700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:39.277266 18700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:39.295434 18700 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38969
I20260812 00:39:39.295469 18846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38969 every 8 connection(s)
I20260812 00:39:39.297079 18700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:39.301412 18847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:39.304088 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 18700
I20260812 00:39:39.304296 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:39.304870 8145 external_mini_cluster.cc:1428] Setting key b3a2f04078732f2e32f2d18687a9c830
I20260812 00:39:39.313583 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:39.306943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40975 (local address 127.0.0.1:38969)
0812 00:39:39.307886 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:39.307900 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:39.307951 (+ 51us) server_negotiation.cc:407] Connection header received
0812 00:39:39.309808 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:39.309836 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:39.310230 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:39.310671 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:39.311804 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:39.311811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:39.311825 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:39.311840 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:39.311914 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:39.311941 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:39.312183 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:39.312336 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:39:39.312545 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":447,"threads_started":1}
I20260812 00:39:39.314798 18847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582: Bootstrap starting.
I20260812 00:39:39.317775 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:39.318650 18847 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:39.319700 18847 log.cc:824] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:39.323031 18847 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582: No bootstrap required, opened a new log
I20260812 00:39:39.324810 18852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:39.322186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35973 (local address 127.0.0.1:38969)
0812 00:39:39.322389 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:39.322395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:39.322757 (+ 362us) server_negotiation.cc:407] Connection header received
0812 00:39:39.322951 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:39.322954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:39.323004 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:39.323090 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:39.324103 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:39.324107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:39.324110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:39.324125 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:39.324168 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:39.324183 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:39.324264 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:39.324458 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 00:39:39.324527 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 00:39:39.331763 18847 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e9adb280a8414bb9704ca8214e4582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38969 } }
I20260812 00:39:39.332213 18847 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:39.332295 18847 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1e9adb280a8414bb9704ca8214e4582, State: Initialized, Role: FOLLOWER
I20260812 00:39:39.333143 18847 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e9adb280a8414bb9704ca8214e4582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38969 } }
I20260812 00:39:39.333570 18847 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:39.333739 18847 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:39.333935 18847 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:39.336400 18847 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e9adb280a8414bb9704ca8214e4582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38969 } }
I20260812 00:39:39.336994 18847 leader_election.cc:302] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1e9adb280a8414bb9704ca8214e4582; no voters:
I20260812 00:39:39.337697 18847 leader_election.cc:288] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:39.337903 18873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:39.339022 18873 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [term 1 LEADER]: Becoming Leader. State: Replica: d1e9adb280a8414bb9704ca8214e4582, State: Running, Role: LEADER
I20260812 00:39:39.339613 18873 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e9adb280a8414bb9704ca8214e4582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38969 } }
I20260812 00:39:39.340039 18847 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:39.342609 18875 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1e9adb280a8414bb9704ca8214e4582. Latest consensus state: current_term: 1 leader_uuid: "d1e9adb280a8414bb9704ca8214e4582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e9adb280a8414bb9704ca8214e4582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38969 } } }
I20260812 00:39:39.343083 18875 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:39.343664 18874 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1e9adb280a8414bb9704ca8214e4582" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1e9adb280a8414bb9704ca8214e4582" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38969 } } }
I20260812 00:39:39.344027 18874 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:39.346145 18884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:39.352319 18884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:39.362659 18884 catalog_manager.cc:1347] Generated new cluster ID: c6171c7328f44f8e9a6e84ed92e4e9a0
I20260812 00:39:39.362808 18884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:39.374130 18884 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:39.374630 18884 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:39.381848 18884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1e9adb280a8414bb9704ca8214e4582: Generated new TSK 0
I20260812 00:39:39.382826 18884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:39.395453 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18700
W20260812 00:39:39.396692 18839 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40975
W20260812 00:39:39.397449 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18700: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:39.397619 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 18700
2026-08-12T00:39:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2249 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:39 dist-test-slave-gjw7 krb5kdc[18908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:39 dist-test-slave-gjw7 krb5kdc[18908](info): set up 2 sockets
Aug 12 00:39:39 dist-test-slave-gjw7 krb5kdc[18908](info): commencing operation
krb5kdc: starting...
W20260812 00:39:40.741240 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.005s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:40 dist-test-slave-gjw7 krb5kdc[18908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:41.075126 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:41.438052 18940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:41.438412 18940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:41.438555 18940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:41.446213 18940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:41.446368 18940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:41.446446 18940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:41.446529 18940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:41.446588 18940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:41.457435 18940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:41.458879 18940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:41.460290 18940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:41.472944 18954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:41.473034 18949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:41.473011 18950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:41.473718 18940 server_base.cc:1034] running on GCE node
I20260812 00:39:41.474673 18940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:41.476337 18940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:41.477597 18940 hybrid_clock.cc:648] HybridClock initialized: now 1786495181477520 us; error 67 us; skew 500 ppm
I20260812 00:39:41.478022 18940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:41 dist-test-slave-gjw7 krb5kdc[18908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495181, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:41.481441 18940 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:41.483129 18940 webserver.cc:460] Webserver started at http://127.0.0.1:45635/ using document root <none> and password file <none>
I20260812 00:39:41.483947 18940 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:41.484083 18940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:41.484422 18940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:41.487840 18940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "9d2c803893464a799499eb5d728850ad"
format_stamp: "Formatted at 2026-08-12 00:39:41 on dist-test-slave-gjw7"
server_key: "0305ce3499eceb18ab7263f5eefefbe8"
server_key_iv: "64dad6bf9ee575a649a804b771ee531e"
server_key_version: "encryptionkey@0"
I20260812 00:39:41.488718 18940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "9d2c803893464a799499eb5d728850ad"
format_stamp: "Formatted at 2026-08-12 00:39:41 on dist-test-slave-gjw7"
server_key: "0305ce3499eceb18ab7263f5eefefbe8"
server_key_iv: "64dad6bf9ee575a649a804b771ee531e"
server_key_version: "encryptionkey@0"
I20260812 00:39:41.494763 18940 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 00:39:41.498762 18970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:41.500344 18940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:39:41.500554 18940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "9d2c803893464a799499eb5d728850ad"
format_stamp: "Formatted at 2026-08-12 00:39:41 on dist-test-slave-gjw7"
server_key: "0305ce3499eceb18ab7263f5eefefbe8"
server_key_iv: "64dad6bf9ee575a649a804b771ee531e"
server_key_version: "encryptionkey@0"
I20260812 00:39:41.500765 18940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:41.534420 18940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:41.539374 18940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:41.539775 18940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:41.560369 18940 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41637
I20260812 00:39:41.560421 19078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41637 every 8 connection(s)
I20260812 00:39:41.562022 18940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:41.566198 19080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:41.568862 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 18940
I20260812 00:39:41.569064 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:41.569658 8145 external_mini_cluster.cc:1428] Setting key 292fe41eb3c6c132815849dfc4d4d1c2
I20260812 00:39:41.579246 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad: Bootstrap starting.
I20260812 00:39:41.582798 19080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:41.583913 19080 log.cc:824] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:39:41 dist-test-slave-gjw7 krb5kdc[18908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 00:39:41.588039 19080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad: No bootstrap required, opened a new log
I20260812 00:39:41.596614 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:41.572090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:44909 (local address 127.0.0.1:41637)
0812 00:39:41.573042 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:41.573057 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:41.573113 (+ 56us) server_negotiation.cc:407] Connection header received
0812 00:39:41.575010 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:41.575039 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:41.575532 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:41.576029 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:41.577781 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:41.579161 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:41.580532 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:41.581279 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:41.581502 (+ 223us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:41.586377 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:41.586392 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:41.586416 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:41.586470 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:41.590776 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:41.591968 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:41.591979 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:41.591994 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:41.592267 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:41.592884 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:41.592890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:41.592894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:41.593771 (+ 877us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:41.594134 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:41.594594 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 00:39:41.595017 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":402,"threads_started":1}
I20260812 00:39:41.598912 19080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2c803893464a799499eb5d728850ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41637 } }
I20260812 00:39:41.599337 19080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:41.599439 19080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d2c803893464a799499eb5d728850ad, State: Initialized, Role: FOLLOWER
I20260812 00:39:41.600066 19080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2c803893464a799499eb5d728850ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41637 } }
I20260812 00:39:41.600381 19080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:41.600538 19080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:41.600726 19080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:41.601439 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:41.603188 19080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2c803893464a799499eb5d728850ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41637 } }
I20260812 00:39:41.603775 19080 leader_election.cc:302] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d2c803893464a799499eb5d728850ad; no voters:
I20260812 00:39:41.604528 19080 leader_election.cc:288] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:41.604598 19091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:41.605947 19091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [term 1 LEADER]: Becoming Leader. State: Replica: 9d2c803893464a799499eb5d728850ad, State: Running, Role: LEADER
I20260812 00:39:41.607012 19091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2c803893464a799499eb5d728850ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41637 } }
I20260812 00:39:41.607252 19080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:41.611567 19097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d2c803893464a799499eb5d728850ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2c803893464a799499eb5d728850ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41637 } } }
I20260812 00:39:41.612042 19097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:41.612658 19102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d2c803893464a799499eb5d728850ad. Latest consensus state: current_term: 1 leader_uuid: "9d2c803893464a799499eb5d728850ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d2c803893464a799499eb5d728850ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41637 } } }
I20260812 00:39:41.613027 19102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:41.625872 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:41.606484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:36001 (local address 127.0.0.1:41637)
0812 00:39:41.606768 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:41.606774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:41.606901 (+ 127us) server_negotiation.cc:407] Connection header received
0812 00:39:41.607080 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:41.607085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:41.607156 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:41.607248 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:41.608904 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:41.609939 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:41.611679 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:41.612377 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:41.612489 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:41.616548 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:41.616557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:41.616563 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:41.616621 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:41.621096 (+ 4475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:41.622600 (+ 1504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:41.622608 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:41.622612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:41.622854 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:41.623525 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:41.623530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:41.623534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:41.624048 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:41.624321 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:41.625157 (+ 836us) server_negotiation.cc:299] Negotiation successful
0812 00:39:41.625438 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260812 00:39:41.630451 19112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:41.636611 19112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:41.646827 19112 catalog_manager.cc:1347] Generated new cluster ID: d99694473737438c9b461270d4338f97
I20260812 00:39:41.646970 19112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:41.655829 19112 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:41.657259 19112 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:41.669433 19112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d2c803893464a799499eb5d728850ad: Generated new TSK 0
I20260812 00:39:41.670356 19112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:41.681912 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 18940
I20260812 00:39:41.683367 19073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:41.775882 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 18940
W20260812 00:39:41.801879 18906 connection.cc:535] client connection to 127.0.0.1:41637 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41637): Connection reset by peer (error 104)
2026-08-12T00:39:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2370 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:41 dist-test-slave-gjw7 krb5kdc[19141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:41 dist-test-slave-gjw7 krb5kdc[19141](info): set up 2 sockets
Aug 12 00:39:41 dist-test-slave-gjw7 krb5kdc[19141](info): commencing operation
krb5kdc: starting...
W20260812 00:39:43.119989 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:43 dist-test-slave-gjw7 krb5kdc[19141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:43.457610 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:43.825728 19170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:43.826062 19170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:43.826159 19170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:43.834233 19170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:43.834389 19170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:43.834448 19170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:43.834511 19170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:43.834556 19170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:43.845014 19170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:43.846414 19170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:43.847749 19170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:43.860368 19178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:43.860404 19179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:43.860440 19184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:43.861106 19170 server_base.cc:1034] running on GCE node
I20260812 00:39:43.861991 19170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:43.863499 19170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:43.864715 19170 hybrid_clock.cc:648] HybridClock initialized: now 1786495183864674 us; error 28 us; skew 500 ppm
I20260812 00:39:43.865070 19170 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:43 dist-test-slave-gjw7 krb5kdc[19141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495183, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:43.872956 19170 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:43.874634 19170 webserver.cc:460] Webserver started at http://127.0.0.1:37061/ using document root <none> and password file <none>
I20260812 00:39:43.875361 19170 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:43.875473 19170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:43.875765 19170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:43.879017 19170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "021c9b51e1d84e8a9c25d05e553df323"
format_stamp: "Formatted at 2026-08-12 00:39:43 on dist-test-slave-gjw7"
server_key: "bfdf78511172221903765d262ca30e44"
server_key_iv: "e00b620b80ccbff2f5b6613be877b962"
server_key_version: "encryptionkey@0"
I20260812 00:39:43.879918 19170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "021c9b51e1d84e8a9c25d05e553df323"
format_stamp: "Formatted at 2026-08-12 00:39:43 on dist-test-slave-gjw7"
server_key: "bfdf78511172221903765d262ca30e44"
server_key_iv: "e00b620b80ccbff2f5b6613be877b962"
server_key_version: "encryptionkey@0"
I20260812 00:39:43.887044 19170 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260812 00:39:43.891674 19196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:43.893507 19170 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 00:39:43.893711 19170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "021c9b51e1d84e8a9c25d05e553df323"
format_stamp: "Formatted at 2026-08-12 00:39:43 on dist-test-slave-gjw7"
server_key: "bfdf78511172221903765d262ca30e44"
server_key_iv: "e00b620b80ccbff2f5b6613be877b962"
server_key_version: "encryptionkey@0"
I20260812 00:39:43.893939 19170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:43.910640 19170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:43.915045 19170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:43.915325 19170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:43.934573 19170 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44555
I20260812 00:39:43.934633 19303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44555 every 8 connection(s)
I20260812 00:39:43.936088 19170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:43.940244 19304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:43.941593 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19170
I20260812 00:39:43.941805 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:43.942366 8145 external_mini_cluster.cc:1428] Setting key 95f5527b3b580833295c770c0689246e
I20260812 00:39:43.956094 19304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323: Bootstrap starting.
I20260812 00:39:43.958250 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:43.944374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:55779 (local address 127.0.0.1:44555)
0812 00:39:43.945256 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:43.945273 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:43.945378 (+ 105us) server_negotiation.cc:407] Connection header received
0812 00:39:43.947388 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:43.947423 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:43.948035 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:43.948558 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:43.949108 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:43.951005 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:43.952423 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:43.953621 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:43.953809 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:43.955123 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:43.955135 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:43.955156 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:43.955935 (+ 779us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:43.956057 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:43.956098 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:43.956258 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:43.956561 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 00:39:43.956766 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":354,"threads_started":1}
I20260812 00:39:43.960361 19304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:43.961818 19304 log.cc:824] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:43.963183 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:43.965512 19304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323: No bootstrap required, opened a new log
I20260812 00:39:43.976547 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:43.968988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:47351 (local address 127.0.0.1:44555)
0812 00:39:43.969187 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:43.969198 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:43.969773 (+ 575us) server_negotiation.cc:407] Connection header received
0812 00:39:43.969987 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:43.969999 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:43.970078 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:43.970175 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:43.970814 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:43.971647 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:43.973729 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:43.974223 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:43.974339 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:43.975733 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:43.975739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:43.975742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:43.975765 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:43.975817 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:43.975836 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:43.975947 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:43.976140 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 00:39:43.976192 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 00:39:43.976440 19304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021c9b51e1d84e8a9c25d05e553df323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } }
I20260812 00:39:43.977002 19304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:43.977137 19304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021c9b51e1d84e8a9c25d05e553df323, State: Initialized, Role: FOLLOWER
I20260812 00:39:43.977866 19304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021c9b51e1d84e8a9c25d05e553df323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } }
I20260812 00:39:43.978176 19304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:43.978333 19304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:43.978504 19304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:43.981106 19304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021c9b51e1d84e8a9c25d05e553df323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } }
I20260812 00:39:43.981729 19304 leader_election.cc:302] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 021c9b51e1d84e8a9c25d05e553df323; no voters:
I20260812 00:39:43.982381 19304 leader_election.cc:288] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:43.982525 19321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:43.983232 19321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [term 1 LEADER]: Becoming Leader. State: Replica: 021c9b51e1d84e8a9c25d05e553df323, State: Running, Role: LEADER
I20260812 00:39:43.983867 19321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021c9b51e1d84e8a9c25d05e553df323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } }
I20260812 00:39:43.984602 19304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:43.986763 19322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "021c9b51e1d84e8a9c25d05e553df323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021c9b51e1d84e8a9c25d05e553df323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } } }
I20260812 00:39:43.987135 19322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:43.987838 19324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 021c9b51e1d84e8a9c25d05e553df323. Latest consensus state: current_term: 1 leader_uuid: "021c9b51e1d84e8a9c25d05e553df323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021c9b51e1d84e8a9c25d05e553df323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } } }
I20260812 00:39:43.988127 19324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:43.988829 19331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:43.995188 19331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:44.006891 19331 catalog_manager.cc:1347] Generated new cluster ID: ba403274a0e043a594e11befe5a4175b
I20260812 00:39:44.007035 19331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:44.018945 19331 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:44.020407 19331 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:44.031342 19331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 021c9b51e1d84e8a9c25d05e553df323: Generated new TSK 0
I20260812 00:39:44.032248 19331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:44.045251 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19170
W20260812 00:39:44.046805 19290 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.206:55779
W20260812 00:39:44.047690 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19170: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:44.047861 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19170
W20260812 00:39:44.072027 19136 connection.cc:535] client connection to 127.0.0.1:44555 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44555): Connection reset by peer (error 104)
2026-08-12T00:39:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2269 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:44 dist-test-slave-gjw7 krb5kdc[19357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:44 dist-test-slave-gjw7 krb5kdc[19357](info): set up 2 sockets
Aug 12 00:39:44 dist-test-slave-gjw7 krb5kdc[19357](info): commencing operation
krb5kdc: starting...
W20260812 00:39:45.400058 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:45 dist-test-slave-gjw7 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:45.734061 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:46.101941 19387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:46.102273 19387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:46.102504 19387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:46.110157 19387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:46.110296 19387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:46.110344 19387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:46.110421 19387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:46.110481 19387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:46.120998 19387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:46.122426 19387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:46.123805 19387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:46.136580 19401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:46.136535 19402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:46.136624 19407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:46.137634 19387 server_base.cc:1034] running on GCE node
I20260812 00:39:46.138566 19387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:46.140177 19387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:46.141453 19387 hybrid_clock.cc:648] HybridClock initialized: now 1786495186141388 us; error 61 us; skew 500 ppm
I20260812 00:39:46.141867 19387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:46 dist-test-slave-gjw7 krb5kdc[19357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495186, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:46.145237 19387 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:46.147075 19387 webserver.cc:460] Webserver started at http://127.0.0.1:43647/ using document root <none> and password file <none>
I20260812 00:39:46.147897 19387 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:46.148051 19387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:46.148386 19387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:46.151885 19387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "386ae684cf0c4e69ae9a06975e0ef12e"
format_stamp: "Formatted at 2026-08-12 00:39:46 on dist-test-slave-gjw7"
server_key: "1fa4735e5b8642692a907a315ac7f5a3"
server_key_iv: "fbc57249b4016b4aa3214aef5cbf0803"
server_key_version: "encryptionkey@0"
I20260812 00:39:46.152825 19387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "386ae684cf0c4e69ae9a06975e0ef12e"
format_stamp: "Formatted at 2026-08-12 00:39:46 on dist-test-slave-gjw7"
server_key: "1fa4735e5b8642692a907a315ac7f5a3"
server_key_iv: "fbc57249b4016b4aa3214aef5cbf0803"
server_key_version: "encryptionkey@0"
I20260812 00:39:46.159055 19387 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260812 00:39:46.163331 19423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:46.164960 19387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 00:39:46.165174 19387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "386ae684cf0c4e69ae9a06975e0ef12e"
format_stamp: "Formatted at 2026-08-12 00:39:46 on dist-test-slave-gjw7"
server_key: "1fa4735e5b8642692a907a315ac7f5a3"
server_key_iv: "fbc57249b4016b4aa3214aef5cbf0803"
server_key_version: "encryptionkey@0"
I20260812 00:39:46.165455 19387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:46.171900 19387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:46.177093 19387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:46.177469 19387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:46.196898 19387 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40441
I20260812 00:39:46.196946 19537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40441 every 8 connection(s)
I20260812 00:39:46.198518 19387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:46.202831 19539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:46.207731 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19387
I20260812 00:39:46.207958 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:46.208534 8145 external_mini_cluster.cc:1428] Setting key 358e597471ac684300ba501b70eddf89
I20260812 00:39:46.217396 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:46.210720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:42825 (local address 127.0.0.1:40441)
0812 00:39:46.211611 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:46.211624 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:46.211671 (+ 47us) server_negotiation.cc:407] Connection header received
0812 00:39:46.213529 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:46.213567 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:46.214156 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:46.214626 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:46.215753 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:46.215759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:46.215774 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:46.215789 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:46.215868 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:46.215892 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:46.216103 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:46.216204 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 00:39:46.216389 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":390,"threads_started":1}
I20260812 00:39:46.218273 19539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e: Bootstrap starting.
I20260812 00:39:46.221561 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:46.222110 19539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:46.223240 19539 log.cc:824] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:46.227214 19539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e: No bootstrap required, opened a new log
I20260812 00:39:46.229285 19543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:46.226451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:37591 (local address 127.0.0.1:40441)
0812 00:39:46.226644 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:46.226650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:46.227031 (+ 381us) server_negotiation.cc:407] Connection header received
0812 00:39:46.227276 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:46.227282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:46.227351 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:46.227463 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:46.228548 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:46.228564 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:46.228569 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:46.228591 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:46.228648 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:46.228670 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:46.228794 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:46.228915 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 00:39:46.228982 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:39:46.236212 19539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ae684cf0c4e69ae9a06975e0ef12e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } }
I20260812 00:39:46.236667 19539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:46.236783 19539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 386ae684cf0c4e69ae9a06975e0ef12e, State: Initialized, Role: FOLLOWER
I20260812 00:39:46.237452 19539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ae684cf0c4e69ae9a06975e0ef12e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } }
I20260812 00:39:46.237703 19539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:46.237815 19539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:46.237962 19539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:46.241122 19539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ae684cf0c4e69ae9a06975e0ef12e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } }
I20260812 00:39:46.241801 19539 leader_election.cc:302] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 386ae684cf0c4e69ae9a06975e0ef12e; no voters:
I20260812 00:39:46.242431 19539 leader_election.cc:288] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:46.242597 19556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:46.243367 19556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [term 1 LEADER]: Becoming Leader. State: Replica: 386ae684cf0c4e69ae9a06975e0ef12e, State: Running, Role: LEADER
I20260812 00:39:46.244048 19556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ae684cf0c4e69ae9a06975e0ef12e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } }
I20260812 00:39:46.244742 19539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:46.247239 19559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "386ae684cf0c4e69ae9a06975e0ef12e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ae684cf0c4e69ae9a06975e0ef12e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } } }
I20260812 00:39:46.247531 19561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 386ae684cf0c4e69ae9a06975e0ef12e. Latest consensus state: current_term: 1 leader_uuid: "386ae684cf0c4e69ae9a06975e0ef12e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "386ae684cf0c4e69ae9a06975e0ef12e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40441 } } }
I20260812 00:39:46.247962 19559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:46.247972 19561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:46.250506 19566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:46.256749 19566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:46.266698 19566 catalog_manager.cc:1347] Generated new cluster ID: c27c6713985243789ff7875ff1618e4f
I20260812 00:39:46.266860 19566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:46.282505 19566 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:46.283267 19566 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:46.299489 19566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 386ae684cf0c4e69ae9a06975e0ef12e: Generated new TSK 0
I20260812 00:39:46.300534 19566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:46.308187 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19387
W20260812 00:39:46.309597 19527 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.206:42825
W20260812 00:39:46.310482 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19387: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:46.310652 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19387
2026-08-12T00:39:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:46 dist-test-slave-gjw7 krb5kdc[19592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:46 dist-test-slave-gjw7 krb5kdc[19592](info): set up 2 sockets
Aug 12 00:39:46 dist-test-slave-gjw7 krb5kdc[19592](info): commencing operation
krb5kdc: starting...
W20260812 00:39:47.679733 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:47 dist-test-slave-gjw7 krb5kdc[19592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:48.033334 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38789
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:48.404445 19612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:48.404917 19612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:48.405087 19612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:48.415385 19612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:48.415552 19612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:48.415633 19612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:48.415725 19612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:48.415791 19612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:48.427078 19612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38789
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:48.428622 19612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:48.430135 19612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:48.442495 19629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:48.442531 19625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:48.442373 19626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:48.443217 19612 server_base.cc:1034] running on GCE node
I20260812 00:39:48.444084 19612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:48.445735 19612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:48.446964 19612 hybrid_clock.cc:648] HybridClock initialized: now 1786495188446896 us; error 57 us; skew 500 ppm
I20260812 00:39:48.447348 19612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:48 dist-test-slave-gjw7 krb5kdc[19592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495188, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:48.450742 19612 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:48.452333 19612 webserver.cc:460] Webserver started at http://127.0.0.1:38513/ using document root <none> and password file <none>
I20260812 00:39:48.453088 19612 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:48.453218 19612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:48.453608 19612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:48.456843 19612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "c7d9972f173e49148963b10a2ebe76e2"
format_stamp: "Formatted at 2026-08-12 00:39:48 on dist-test-slave-gjw7"
server_key: "4a93dced1105e686605edef3e26ae680"
server_key_iv: "18966c1114105e9f5cddabfaaaf95262"
server_key_version: "encryptionkey@0"
I20260812 00:39:48.457744 19612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "c7d9972f173e49148963b10a2ebe76e2"
format_stamp: "Formatted at 2026-08-12 00:39:48 on dist-test-slave-gjw7"
server_key: "4a93dced1105e686605edef3e26ae680"
server_key_iv: "18966c1114105e9f5cddabfaaaf95262"
server_key_version: "encryptionkey@0"
I20260812 00:39:48.463374 19612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:39:48.467447 19649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:48.469007 19612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 00:39:48.469197 19612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "c7d9972f173e49148963b10a2ebe76e2"
format_stamp: "Formatted at 2026-08-12 00:39:48 on dist-test-slave-gjw7"
server_key: "4a93dced1105e686605edef3e26ae680"
server_key_iv: "18966c1114105e9f5cddabfaaaf95262"
server_key_version: "encryptionkey@0"
I20260812 00:39:48.469488 19612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:48.486308 19612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:48.491163 19612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:48.491475 19612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:48.511042 19612 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38789
I20260812 00:39:48.511071 19748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38789 every 8 connection(s)
I20260812 00:39:48.512559 19612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:48.516670 19749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:48.517251 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19612
I20260812 00:39:48.517448 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:48.517900 8145 external_mini_cluster.cc:1428] Setting key 60b9f6c73b2fccac4a74f4d9c840ccaa
I20260812 00:39:48.532572 19749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2: Bootstrap starting.
Aug 12 00:39:48 dist-test-slave-gjw7 krb5kdc[19592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 00:39:48.537034 19749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:48.538472 19749 log.cc:824] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:48.542284 19749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2: No bootstrap required, opened a new log
I20260812 00:39:48.545787 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:48.519883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:58001 (local address 127.0.0.1:38789)
0812 00:39:48.520906 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:48.520924 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:48.521322 (+ 398us) server_negotiation.cc:407] Connection header received
0812 00:39:48.523560 (+ 2238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:48.523595 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:48.524132 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:48.524538 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:48.526590 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:48.528089 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:48.530186 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:48.531046 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:48.531246 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:48.536914 (+ 5668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:48.536925 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:48.536946 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:48.536997 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:48.540817 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:48.541997 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:48.542004 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:48.542012 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:48.542199 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:48.542731 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:48.542734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:48.542736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:48.543322 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:48.543639 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:48.544138 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 00:39:48.544493 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":412,"threads_started":1}
I20260812 00:39:48.550261 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:48.551393 19749 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d9972f173e49148963b10a2ebe76e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } }
I20260812 00:39:48.551901 19749 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:48.552003 19749 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d9972f173e49148963b10a2ebe76e2, State: Initialized, Role: FOLLOWER
I20260812 00:39:48.552734 19749 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d9972f173e49148963b10a2ebe76e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } }
I20260812 00:39:48.553067 19749 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:48.553224 19749 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:48.553447 19749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:48.556555 19749 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d9972f173e49148963b10a2ebe76e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } }
I20260812 00:39:48.557242 19749 leader_election.cc:302] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7d9972f173e49148963b10a2ebe76e2; no voters:
I20260812 00:39:48.557960 19749 leader_election.cc:288] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:48.558142 19774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:48.559316 19774 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [term 1 LEADER]: Becoming Leader. State: Replica: c7d9972f173e49148963b10a2ebe76e2, State: Running, Role: LEADER
I20260812 00:39:48.560035 19774 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d9972f173e49148963b10a2ebe76e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } }
I20260812 00:39:48.560366 19749 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:48.566629 19778 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7d9972f173e49148963b10a2ebe76e2. Latest consensus state: current_term: 1 leader_uuid: "c7d9972f173e49148963b10a2ebe76e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d9972f173e49148963b10a2ebe76e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } } }
I20260812 00:39:48.567054 19778 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:48.566378 19776 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7d9972f173e49148963b10a2ebe76e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d9972f173e49148963b10a2ebe76e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38789 } } }
I20260812 00:39:48.568324 19776 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:48.576515 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:48.555779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:54707 (local address 127.0.0.1:38789)
0812 00:39:48.555994 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:48.556000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:48.556595 (+ 595us) server_negotiation.cc:407] Connection header received
0812 00:39:48.556739 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:48.556745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:48.556841 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:48.556968 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:48.559131 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:48.560328 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:48.562426 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:48.563209 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:48.563327 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:48.567263 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:48.567272 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:48.567277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:48.567324 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:48.571617 (+ 4293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:48.573074 (+ 1457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:48.573085 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:48.573089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:48.573316 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:48.574000 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:48.574151 (+ 151us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:48.574156 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:48.574658 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:48.574963 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:48.575792 (+ 829us) server_negotiation.cc:299] Negotiation successful
0812 00:39:48.576079 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:39:48.579834 19790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:48.586239 19790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:48.596997 19790 catalog_manager.cc:1347] Generated new cluster ID: df1918b1250e4a1a9efcbc7e21102e85
I20260812 00:39:48.597148 19790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:48.626005 19790 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:48.628039 19790 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:48.637707 19790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7d9972f173e49148963b10a2ebe76e2: Generated new TSK 0
I20260812 00:39:48.638625 19790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:48.649343 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19612
I20260812 00:39:48.651185 19741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:48.744477 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19612
W20260812 00:39:48.772672 19588 connection.cc:535] client connection to 127.0.0.1:38789 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38789): Connection reset by peer (error 104)
2026-08-12T00:39:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2426 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:48 dist-test-slave-gjw7 krb5kdc[19809](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:48 dist-test-slave-gjw7 krb5kdc[19809](info): set up 2 sockets
Aug 12 00:39:48 dist-test-slave-gjw7 krb5kdc[19809](info): commencing operation
krb5kdc: starting...
W20260812 00:39:50.088841 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:50 dist-test-slave-gjw7 krb5kdc[19809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:50.421887 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:50.792290 19841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:50.792642 19841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:50.792771 19841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:50.800693 19841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:50.800848 19841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:50.800895 19841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:50.800985 19841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:50.801050 19841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:50.812274 19841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:50.813835 19841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:50.815363 19841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:50.828794 19858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:50.828886 19853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:50.828864 19855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:50.830302 19841 server_base.cc:1034] running on GCE node
I20260812 00:39:50.831259 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:50.832901 19841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:50.834164 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786495190834107 us; error 56 us; skew 500 ppm
I20260812 00:39:50.834631 19841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:50 dist-test-slave-gjw7 krb5kdc[19809](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495190, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:50.841468 19841 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:50.843120 19841 webserver.cc:460] Webserver started at http://127.0.0.1:42627/ using document root <none> and password file <none>
I20260812 00:39:50.843892 19841 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:50.844031 19841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:50.844348 19841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:50.847641 19841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "c9da518446804ae880c3837dff4714df"
format_stamp: "Formatted at 2026-08-12 00:39:50 on dist-test-slave-gjw7"
server_key: "525960cd3150731677d016ca8658569f"
server_key_iv: "cfb7018ca1b4b3cda24171529cdb4826"
server_key_version: "encryptionkey@0"
I20260812 00:39:50.848500 19841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "c9da518446804ae880c3837dff4714df"
format_stamp: "Formatted at 2026-08-12 00:39:50 on dist-test-slave-gjw7"
server_key: "525960cd3150731677d016ca8658569f"
server_key_iv: "cfb7018ca1b4b3cda24171529cdb4826"
server_key_version: "encryptionkey@0"
I20260812 00:39:50.854298 19841 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 00:39:50.858758 19871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:50.860468 19841 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 00:39:50.860706 19841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "c9da518446804ae880c3837dff4714df"
format_stamp: "Formatted at 2026-08-12 00:39:50 on dist-test-slave-gjw7"
server_key: "525960cd3150731677d016ca8658569f"
server_key_iv: "cfb7018ca1b4b3cda24171529cdb4826"
server_key_version: "encryptionkey@0"
I20260812 00:39:50.860944 19841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:50.888597 19841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:50.892895 19841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:50.893195 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:50.912439 19841 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39241
I20260812 00:39:50.912482 19979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39241 every 8 connection(s)
I20260812 00:39:50.913998 19841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:50.915881 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 19841
I20260812 00:39:50.916064 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:50.916527 8145 external_mini_cluster.cc:1428] Setting key 78734ae71b7a593c5dfa3ce0ac727cb5
I20260812 00:39:50.918892 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:50.933153 19984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:50.918605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:34703 (local address 127.0.0.1:39241)
0812 00:39:50.919491 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:50.919509 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:50.919564 (+ 55us) server_negotiation.cc:407] Connection header received
0812 00:39:50.922060 (+ 2496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:50.922094 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:50.922744 (+ 650us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:50.923265 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:50.923863 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:50.926020 (+ 2157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:50.927552 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:50.928613 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:50.928809 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:50.930105 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:50.930116 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:50.930136 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:50.930957 (+ 821us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:50.931068 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:50.931107 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:50.931296 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:50.931538 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 00:39:50.931755 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":352,"threads_started":1}
I20260812 00:39:50.934307 19980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df: Bootstrap starting.
I20260812 00:39:50.937711 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:50.938006 19980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:50.939424 19980 log.cc:824] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:50.943354 19980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df: No bootstrap required, opened a new log
I20260812 00:39:50.950580 19984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:50.942973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:37137 (local address 127.0.0.1:39241)
0812 00:39:50.943187 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:50.943194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:50.943701 (+ 507us) server_negotiation.cc:407] Connection header received
0812 00:39:50.943930 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:50.943934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:50.944010 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:50.944097 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:50.944818 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:50.945722 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:50.947681 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:50.948183 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:50.948285 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:50.949646 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:50.949652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:50.949656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:50.949684 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:50.949761 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:50.949785 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:50.949911 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:50.950151 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 00:39:50.950223 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 00:39:50.953127 19980 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9da518446804ae880c3837dff4714df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260812 00:39:50.953660 19980 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:50.953769 19980 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9da518446804ae880c3837dff4714df, State: Initialized, Role: FOLLOWER
I20260812 00:39:50.954389 19980 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9da518446804ae880c3837dff4714df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260812 00:39:50.954623 19980 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:50.954764 19980 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:50.954921 19980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:50.957336 19980 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9da518446804ae880c3837dff4714df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260812 00:39:50.957949 19980 leader_election.cc:302] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9da518446804ae880c3837dff4714df; no voters:
I20260812 00:39:50.958678 19980 leader_election.cc:288] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:50.958824 19994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:50.959610 19994 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [term 1 LEADER]: Becoming Leader. State: Replica: c9da518446804ae880c3837dff4714df, State: Running, Role: LEADER
I20260812 00:39:50.960238 19994 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9da518446804ae880c3837dff4714df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } }
I20260812 00:39:50.960958 19980 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:50.963323 19996 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9da518446804ae880c3837dff4714df. Latest consensus state: current_term: 1 leader_uuid: "c9da518446804ae880c3837dff4714df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9da518446804ae880c3837dff4714df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } } }
I20260812 00:39:50.963348 19995 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9da518446804ae880c3837dff4714df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9da518446804ae880c3837dff4714df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39241 } } }
I20260812 00:39:50.963819 19996 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:50.963879 19995 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:50.970132 20015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:50.976629 20015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:50.987018 20015 catalog_manager.cc:1347] Generated new cluster ID: 177a28a9139340ac98cf34d7c0d357a7
I20260812 00:39:50.987156 20015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:51.009919 20015 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:51.011363 20015 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:51.026521 20015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9da518446804ae880c3837dff4714df: Generated new TSK 0
I20260812 00:39:51.027519 20015 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:51.033630 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19841
W20260812 00:39:51.036163 19971 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.206:34703
W20260812 00:39:51.037207 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 19841: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:51.037424 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 19841
W20260812 00:39:51.064265 19807 connection.cc:535] client connection to 127.0.0.1:39241 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39241): Connection reset by peer (error 104)
2026-08-12T00:39:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:51 dist-test-slave-gjw7 krb5kdc[20039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:51 dist-test-slave-gjw7 krb5kdc[20039](info): set up 2 sockets
Aug 12 00:39:51 dist-test-slave-gjw7 krb5kdc[20039](info): commencing operation
krb5kdc: starting...
W20260812 00:39:52.369977 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.001s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:52 dist-test-slave-gjw7 krb5kdc[20039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 00:39:52.700912 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:53.064975 20063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:53.065357 20063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:53.065518 20063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:53.073069 20063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:53.073201 20063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:53.073282 20063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:53.073395 20063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:53.073467 20063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:53.084084 20063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:53.085592 20063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:53.087037 20063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:53.099633 20071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:53.099610 20075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:53.099572 20073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:53.100368 20063 server_base.cc:1034] running on GCE node
I20260812 00:39:53.101326 20063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:53.102945 20063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:53.104158 20063 hybrid_clock.cc:648] HybridClock initialized: now 1786495193104106 us; error 45 us; skew 500 ppm
I20260812 00:39:53.104534 20063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:53 dist-test-slave-gjw7 krb5kdc[20039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:53.107924 20063 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 00:39:53.109529 20063 webserver.cc:460] Webserver started at http://127.0.0.1:43353/ using document root <none> and password file <none>
I20260812 00:39:53.110248 20063 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:53.110381 20063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:53.110695 20063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:53.114074 20063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "d0d4b16374a74eb6a8ae078d0d283e20"
format_stamp: "Formatted at 2026-08-12 00:39:53 on dist-test-slave-gjw7"
server_key: "803e90895a7172124825abd750154b97"
server_key_iv: "c59082a2b178f4de9391f1ae8fd67fac"
server_key_version: "encryptionkey@0"
I20260812 00:39:53.114984 20063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "d0d4b16374a74eb6a8ae078d0d283e20"
format_stamp: "Formatted at 2026-08-12 00:39:53 on dist-test-slave-gjw7"
server_key: "803e90895a7172124825abd750154b97"
server_key_iv: "c59082a2b178f4de9391f1ae8fd67fac"
server_key_version: "encryptionkey@0"
I20260812 00:39:53.121055 20063 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 00:39:53.125248 20088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:53.126935 20063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:39:53.127142 20063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "d0d4b16374a74eb6a8ae078d0d283e20"
format_stamp: "Formatted at 2026-08-12 00:39:53 on dist-test-slave-gjw7"
server_key: "803e90895a7172124825abd750154b97"
server_key_iv: "c59082a2b178f4de9391f1ae8fd67fac"
server_key_version: "encryptionkey@0"
I20260812 00:39:53.127365 20063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:53.133704 20063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:53.138352 20063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:53.138681 20063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:53.157739 20063 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45079
I20260812 00:39:53.157783 20195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45079 every 8 connection(s)
I20260812 00:39:53.159291 20063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:53.163340 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20063
I20260812 00:39:53.163419 20196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:53.163533 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:53.163973 8145 external_mini_cluster.cc:1428] Setting key aa14baa3705b5838620f81fd7a3f61bd
I20260812 00:39:53.173592 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:53.165984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:47515 (local address 127.0.0.1:45079)
0812 00:39:53.166953 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:53.166973 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:53.167030 (+ 57us) server_negotiation.cc:407] Connection header received
0812 00:39:53.168837 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:53.168883 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:53.169445 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:53.169977 (+ 532us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:53.171052 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:53.171065 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:53.171084 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:53.171105 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:53.171190 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:53.171220 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:53.171451 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:53.171721 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 00:39:53.172175 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":414,"threads_started":1}
I20260812 00:39:53.178130 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:53.179134 20196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20: Bootstrap starting.
I20260812 00:39:53.182797 20196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:53.184368 20196 log.cc:824] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:53.185774 20204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:53.182919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:37687 (local address 127.0.0.1:45079)
0812 00:39:53.183134 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:53.183140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:53.183679 (+ 539us) server_negotiation.cc:407] Connection header received
0812 00:39:53.183833 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:53.183839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:53.183917 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:53.184049 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:53.185054 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:53.185059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:53.185062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:53.185082 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:53.185127 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:53.185144 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:53.185258 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:53.185422 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 00:39:53.185492 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 00:39:53.187785 20196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20: No bootstrap required, opened a new log
I20260812 00:39:53.195669 20196 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d4b16374a74eb6a8ae078d0d283e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45079 } }
I20260812 00:39:53.196118 20196 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:53.196224 20196 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d4b16374a74eb6a8ae078d0d283e20, State: Initialized, Role: FOLLOWER
I20260812 00:39:53.196828 20196 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d4b16374a74eb6a8ae078d0d283e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45079 } }
I20260812 00:39:53.197060 20196 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:53.197168 20196 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:53.197361 20196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:53.199864 20196 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d4b16374a74eb6a8ae078d0d283e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45079 } }
I20260812 00:39:53.200356 20196 leader_election.cc:302] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0d4b16374a74eb6a8ae078d0d283e20; no voters:
I20260812 00:39:53.200951 20196 leader_election.cc:288] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:53.201098 20213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:53.201766 20213 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [term 1 LEADER]: Becoming Leader. State: Replica: d0d4b16374a74eb6a8ae078d0d283e20, State: Running, Role: LEADER
I20260812 00:39:53.202323 20213 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d4b16374a74eb6a8ae078d0d283e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45079 } }
I20260812 00:39:53.203063 20196 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:53.205330 20218 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0d4b16374a74eb6a8ae078d0d283e20. Latest consensus state: current_term: 1 leader_uuid: "d0d4b16374a74eb6a8ae078d0d283e20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d4b16374a74eb6a8ae078d0d283e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45079 } } }
I20260812 00:39:53.205843 20218 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:53.206436 20216 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0d4b16374a74eb6a8ae078d0d283e20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0d4b16374a74eb6a8ae078d0d283e20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45079 } } }
I20260812 00:39:53.206729 20216 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:53.208249 20237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:53.213809 20237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:53.224581 20237 catalog_manager.cc:1347] Generated new cluster ID: c41615c5c89640f3b734d66570f1b10a
I20260812 00:39:53.224731 20237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:53.257887 20237 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:53.258397 20237 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:53.266009 20237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0d4b16374a74eb6a8ae078d0d283e20: Generated new TSK 0
I20260812 00:39:53.267207 20237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:53.280274 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20063
W20260812 00:39:53.281602 20190 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.206:47515
W20260812 00:39:53.282371 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20063: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:53.282533 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20063
2026-08-12T00:39:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2253 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:53 dist-test-slave-gjw7 krb5kdc[20259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:53 dist-test-slave-gjw7 krb5kdc[20259](info): set up 2 sockets
Aug 12 00:39:53 dist-test-slave-gjw7 krb5kdc[20259](info): commencing operation
krb5kdc: starting...
W20260812 00:39:54.629523 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:54 dist-test-slave-gjw7 krb5kdc[20259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:39:54.958467 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:45423
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:40177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:45423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:55.322036 20287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:55.322386 20287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:55.322540 20287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:55.330134 20287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:55.330293 20287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:55.330387 20287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:55.330469 20287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:55.330534 20287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:55.341022 20287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:40177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:45423
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:45423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:55.342507 20287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:55.343892 20287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:55.356529 20305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:55.356633 20303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:55.356602 20300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:55.357208 20287 server_base.cc:1034] running on GCE node
I20260812 00:39:55.358186 20287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:55.359823 20287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:55.361064 20287 hybrid_clock.cc:648] HybridClock initialized: now 1786495195361009 us; error 57 us; skew 500 ppm
I20260812 00:39:55.361537 20287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:55 dist-test-slave-gjw7 krb5kdc[20259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:55.365041 20287 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:39:55.366798 20287 webserver.cc:460] Webserver started at http://127.7.244.126:34441/ using document root <none> and password file <none>
I20260812 00:39:55.367542 20287 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:55.367674 20287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:55.367995 20287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:55.371359 20287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "ec57bb835afa4da084380b76c270eb6c"
format_stamp: "Formatted at 2026-08-12 00:39:55 on dist-test-slave-gjw7"
server_key: "c8f043c649fb22518947deb2ac11cf59"
server_key_iv: "a811e68ede23e75d9539f27d8b8ba455"
server_key_version: "encryptionkey@0"
I20260812 00:39:55.372246 20287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "ec57bb835afa4da084380b76c270eb6c"
format_stamp: "Formatted at 2026-08-12 00:39:55 on dist-test-slave-gjw7"
server_key: "c8f043c649fb22518947deb2ac11cf59"
server_key_iv: "a811e68ede23e75d9539f27d8b8ba455"
server_key_version: "encryptionkey@0"
I20260812 00:39:55.378271 20287 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260812 00:39:55.382323 20315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:55.384119 20287 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 00:39:55.384346 20287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "ec57bb835afa4da084380b76c270eb6c"
format_stamp: "Formatted at 2026-08-12 00:39:55 on dist-test-slave-gjw7"
server_key: "c8f043c649fb22518947deb2ac11cf59"
server_key_iv: "a811e68ede23e75d9539f27d8b8ba455"
server_key_version: "encryptionkey@0"
I20260812 00:39:55.384562 20287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:55.409273 20287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:55.414255 20287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:55.414578 20287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:55.434405 20287 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:45423
I20260812 00:39:55.434419 20415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:45423 every 8 connection(s)
I20260812 00:39:55.436017 20287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:55.440143 20420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:55.442085 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20287
I20260812 00:39:55.442255 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:55.442699 8145 external_mini_cluster.cc:1428] Setting key e2da69ec63d1087ba36df498863be573
I20260812 00:39:55.455066 20420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c: Bootstrap starting.
Aug 12 00:39:55 dist-test-slave-gjw7 krb5kdc[20259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:39:55.459066 20420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:55.460216 20420 log.cc:824] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:55.463838 20420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c: No bootstrap required, opened a new log
I20260812 00:39:55.468953 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:55.444737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53873 (local address 127.7.244.126:45423)
0812 00:39:55.445671 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:55.445683 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:55.445735 (+ 52us) server_negotiation.cc:407] Connection header received
0812 00:39:55.447404 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:55.447435 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:55.447900 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:55.448271 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:55.450296 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:55.452188 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:55.454337 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:55.455136 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:55.455341 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:55.460416 (+ 5075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:55.460428 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:55.460448 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:55.460501 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:55.463899 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:55.464911 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:55.464919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:55.464927 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:55.465126 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:55.465712 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:55.465716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:55.465718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:55.466351 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:55.466724 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:55.467372 (+ 648us) server_negotiation.cc:299] Negotiation successful
0812 00:39:55.467730 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":438,"threads_started":1}
I20260812 00:39:55.472457 20420 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec57bb835afa4da084380b76c270eb6c" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45423 } }
I20260812 00:39:55.472947 20420 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:55.473057 20420 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec57bb835afa4da084380b76c270eb6c, State: Initialized, Role: FOLLOWER
I20260812 00:39:55.473623 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:55.473752 20420 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec57bb835afa4da084380b76c270eb6c" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45423 } }
I20260812 00:39:55.474035 20420 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:55.474181 20420 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:55.474340 20420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:55.476693 20420 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec57bb835afa4da084380b76c270eb6c" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45423 } }
I20260812 00:39:55.477330 20420 leader_election.cc:302] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec57bb835afa4da084380b76c270eb6c; no voters:
I20260812 00:39:55.478060 20420 leader_election.cc:288] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:55.478161 20434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:55.479378 20434 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [term 1 LEADER]: Becoming Leader. State: Replica: ec57bb835afa4da084380b76c270eb6c, State: Running, Role: LEADER
I20260812 00:39:55.480051 20434 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec57bb835afa4da084380b76c270eb6c" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45423 } }
I20260812 00:39:55.480705 20420 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:55.483215 20437 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec57bb835afa4da084380b76c270eb6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec57bb835afa4da084380b76c270eb6c" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45423 } } }
I20260812 00:39:55.483678 20437 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:55.486040 20442 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec57bb835afa4da084380b76c270eb6c. Latest consensus state: current_term: 1 leader_uuid: "ec57bb835afa4da084380b76c270eb6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec57bb835afa4da084380b76c270eb6c" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 45423 } } }
I20260812 00:39:55.486550 20442 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:55.488183 20447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:55.495678 20447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:55.514130 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:55.490232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47651 (local address 127.7.244.126:45423)
0812 00:39:55.490413 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:55.490421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:55.490857 (+ 436us) server_negotiation.cc:407] Connection header received
0812 00:39:55.491049 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:55.491055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:55.491153 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:55.491266 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:55.493557 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:55.494810 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:55.496839 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:55.497603 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:55.497727 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:55.501594 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:55.501602 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:55.501608 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:39:55.501672 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:55.509100 (+ 7428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:55.510386 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:55.510392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:55.510396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:55.510581 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:39:55.511328 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:39:55.511334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:39:55.511338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:39:55.511828 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:55.512145 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:55.513476 (+ 1331us) server_negotiation.cc:299] Negotiation successful
0812 00:39:55.513710 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 00:39:55.518241 20447 catalog_manager.cc:1347] Generated new cluster ID: 919bd2ef0baa4e7fa45e919d7bd903ea
I20260812 00:39:55.518361 20447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:55.527168 20447 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:55.528473 20447 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:55.542407 20447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec57bb835afa4da084380b76c270eb6c: Generated new TSK 0
I20260812 00:39:55.543394 20447 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:55.551749 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20287
I20260812 00:39:55.553010 20407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:39:55.645020 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20287
W20260812 00:39:55.677240 20257 connection.cc:535] client connection to 127.7.244.126:45423 recv error: Network error: failed to read from TLS socket (remote: 127.7.244.126:45423): Connection reset by peer (error 104)
2026-08-12T00:39:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:55 dist-test-slave-gjw7 krb5kdc[20486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:55 dist-test-slave-gjw7 krb5kdc[20486](info): set up 2 sockets
Aug 12 00:39:55 dist-test-slave-gjw7 krb5kdc[20486](info): commencing operation
krb5kdc: starting...
W20260812 00:39:56.984051 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:57 dist-test-slave-gjw7 krb5kdc[20486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:39:57.316545 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42075
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:34009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:42075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:39:57.684048 20511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:39:57.684396 20511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:39:57.684523 20511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:39:57.692117 20511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:39:57.692245 20511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:39:57.692327 20511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:39:57.692427 20511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:39:57.692490 20511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:39:57.703056 20511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:34009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:42075
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:42075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:39:57.704519 20511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:39:57.705914 20511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:39:57.718575 20527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:57.718664 20529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:39:57.718619 20535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:39:57.719574 20511 server_base.cc:1034] running on GCE node
I20260812 00:39:57.720517 20511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:39:57.722223 20511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:39:57.723455 20511 hybrid_clock.cc:648] HybridClock initialized: now 1786495197723417 us; error 73 us; skew 500 ppm
I20260812 00:39:57.723843 20511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:39:57 dist-test-slave-gjw7 krb5kdc[20486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:39:57.732043 20511 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:39:57.733868 20511 webserver.cc:460] Webserver started at http://127.7.244.126:35339/ using document root <none> and password file <none>
I20260812 00:39:57.734685 20511 fs_manager.cc:359] Metadata directory not provided
I20260812 00:39:57.734813 20511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:39:57.735138 20511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:39:57.738606 20511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "b0ab2de7078b4b5c896ffe09bcde66b1"
format_stamp: "Formatted at 2026-08-12 00:39:57 on dist-test-slave-gjw7"
server_key: "b1508bf1840bccc04d31d6884c778e18"
server_key_iv: "072b58b2feb6968a4e29f8c0d6e98c68"
server_key_version: "encryptionkey@0"
I20260812 00:39:57.739462 20511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "b0ab2de7078b4b5c896ffe09bcde66b1"
format_stamp: "Formatted at 2026-08-12 00:39:57 on dist-test-slave-gjw7"
server_key: "b1508bf1840bccc04d31d6884c778e18"
server_key_iv: "072b58b2feb6968a4e29f8c0d6e98c68"
server_key_version: "encryptionkey@0"
I20260812 00:39:57.745193 20511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 00:39:57.749261 20546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:39:57.750836 20511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 00:39:57.751044 20511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "b0ab2de7078b4b5c896ffe09bcde66b1"
format_stamp: "Formatted at 2026-08-12 00:39:57 on dist-test-slave-gjw7"
server_key: "b1508bf1840bccc04d31d6884c778e18"
server_key_iv: "072b58b2feb6968a4e29f8c0d6e98c68"
server_key_version: "encryptionkey@0"
I20260812 00:39:57.751269 20511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:39:57.783334 20511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:39:57.789274 20511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:39:57.789659 20511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:39:57.819980 20511 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:42075
I20260812 00:39:57.819984 20640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:42075 every 8 connection(s)
I20260812 00:39:57.821933 20511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:39:57.824079 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20511
I20260812 00:39:57.824271 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:39:57.824836 8145 external_mini_cluster.cc:1428] Setting key 9b7aa1dbae21e6ea671bfca2665da432
I20260812 00:39:57.827741 20644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:39:57.843554 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:57.826847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56941 (local address 127.7.244.126:42075)
0812 00:39:57.827766 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:57.827782 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:57.827836 (+ 54us) server_negotiation.cc:407] Connection header received
0812 00:39:57.830203 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:57.830236 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:57.830773 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:57.831997 (+ 1224us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:57.832560 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:57.834711 (+ 2151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:57.836790 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:57.837847 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:57.838748 (+ 901us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:57.840043 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:57.840054 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:57.840075 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:57.840903 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:57.841006 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:57.841040 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:57.841778 (+ 738us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:57.841946 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 00:39:57.842147 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":371,"threads_started":1}
I20260812 00:39:57.843564 20644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1: Bootstrap starting.
I20260812 00:39:57.847132 20644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1: Neither blocks nor log segments found. Creating new log.
I20260812 00:39:57.847843 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:39:57.848382 20644 log.cc:824] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1: Log is configured to *not* fsync() on all Append() calls
I20260812 00:39:57.852237 20644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1: No bootstrap required, opened a new log
I20260812 00:39:57.861820 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:39:57.854251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44341 (local address 127.7.244.126:42075)
0812 00:39:57.854510 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 00:39:57.854517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:39:57.854853 (+ 336us) server_negotiation.cc:407] Connection header received
0812 00:39:57.855091 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:39:57.855095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:39:57.855165 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:39:57.855262 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:39:57.855970 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:57.856833 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:57.858819 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:39:57.859351 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:39:57.859483 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:39:57.860930 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:39:57.860936 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:39:57.860940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:39:57.860962 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:39:57.861023 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:39:57.861041 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:39:57.861148 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 00:39:57.861405 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 00:39:57.861465 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:39:57.863034 20644 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab2de7078b4b5c896ffe09bcde66b1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42075 } }
I20260812 00:39:57.863559 20644 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:39:57.863670 20644 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ab2de7078b4b5c896ffe09bcde66b1, State: Initialized, Role: FOLLOWER
I20260812 00:39:57.864286 20644 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab2de7078b4b5c896ffe09bcde66b1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42075 } }
I20260812 00:39:57.864517 20644 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:39:57.864650 20644 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:39:57.864816 20644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:39:57.867405 20644 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab2de7078b4b5c896ffe09bcde66b1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42075 } }
I20260812 00:39:57.868021 20644 leader_election.cc:302] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0ab2de7078b4b5c896ffe09bcde66b1; no voters:
I20260812 00:39:57.868719 20644 leader_election.cc:288] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:39:57.868845 20669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:39:57.870846 20644 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:39:57.873780 20669 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [term 1 LEADER]: Becoming Leader. State: Replica: b0ab2de7078b4b5c896ffe09bcde66b1, State: Running, Role: LEADER
I20260812 00:39:57.874480 20669 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab2de7078b4b5c896ffe09bcde66b1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42075 } }
I20260812 00:39:57.878027 20671 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0ab2de7078b4b5c896ffe09bcde66b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab2de7078b4b5c896ffe09bcde66b1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42075 } } }
I20260812 00:39:57.878970 20671 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [sys.catalog]: This master's current role is: LEADER
I20260812 00:39:57.878151 20675 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0ab2de7078b4b5c896ffe09bcde66b1. Latest consensus state: current_term: 1 leader_uuid: "b0ab2de7078b4b5c896ffe09bcde66b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ab2de7078b4b5c896ffe09bcde66b1" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 42075 } } }
I20260812 00:39:57.879981 20675 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1 [sys.catalog]: This master's current role is: LEADER
W20260812 00:39:57.888046 20704 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:39:57.888238 20704 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:39:57.888439 20708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:39:57.890373 20708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:39:57.899875 20708 catalog_manager.cc:1347] Generated new cluster ID: 85eb24c17014480aa23c50905c898354
I20260812 00:39:57.900012 20708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:39:57.935372 20708 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:39:57.936821 20708 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:39:57.948465 20708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0ab2de7078b4b5c896ffe09bcde66b1: Generated new TSK 0
I20260812 00:39:57.949512 20708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:39:57.965415 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20511
W20260812 00:39:57.966930 20635 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56941
W20260812 00:39:57.967882 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20511: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:39:57.968029 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20511
W20260812 00:39:58.002712 20482 connection.cc:535] client connection to 127.7.244.126:42075 recv error: Network error: failed to read from TLS socket (remote: 127.7.244.126:42075): Connection reset by peer (error 104)
2026-08-12T00:39:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:39:58 dist-test-slave-gjw7 krb5kdc[20718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:39:58 dist-test-slave-gjw7 krb5kdc[20718](info): set up 2 sockets
Aug 12 00:39:58 dist-test-slave-gjw7 krb5kdc[20718](info): commencing operation
krb5kdc: starting...
W20260812 00:39:59.318714 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:39:59 dist-test-slave-gjw7 krb5kdc[20718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:39:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:39:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:39:59.653998 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41625
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:42835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:41625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:40:00.025764 20740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:00.026155 20740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:00.026342 20740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:00.034677 20740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:00.034847 20740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:00.034950 20740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:00.035044 20740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:00.035106 20740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:00.045833 20740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:42835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:41625
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:00.047575 20740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:00.049178 20740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:00.062839 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:00.062973 20754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:00.062970 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:00.063762 20740 server_base.cc:1034] running on GCE node
I20260812 00:40:00.064656 20740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:00.066352 20740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:00.067613 20740 hybrid_clock.cc:648] HybridClock initialized: now 1786495200067558 us; error 45 us; skew 500 ppm
I20260812 00:40:00.068023 20740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:00 dist-test-slave-gjw7 krb5kdc[20718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:00.071525 20740 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:40:00.073200 20740 webserver.cc:460] Webserver started at http://127.7.244.126:44927/ using document root <none> and password file <none>
I20260812 00:40:00.074012 20740 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:00.074151 20740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:00.074518 20740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:00.078020 20740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "1ae5f45170834297adea53788be9048e"
format_stamp: "Formatted at 2026-08-12 00:40:00 on dist-test-slave-gjw7"
server_key: "5368f836ef3bcc1dd0681d1db9cc7c3a"
server_key_iv: "f7357a72d635f396efbcd95c8d745da2"
server_key_version: "encryptionkey@0"
I20260812 00:40:00.078907 20740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "1ae5f45170834297adea53788be9048e"
format_stamp: "Formatted at 2026-08-12 00:40:00 on dist-test-slave-gjw7"
server_key: "5368f836ef3bcc1dd0681d1db9cc7c3a"
server_key_iv: "f7357a72d635f396efbcd95c8d745da2"
server_key_version: "encryptionkey@0"
I20260812 00:40:00.084772 20740 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 00:40:00.088788 20768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:00.090339 20740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:40:00.090595 20740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "1ae5f45170834297adea53788be9048e"
format_stamp: "Formatted at 2026-08-12 00:40:00 on dist-test-slave-gjw7"
server_key: "5368f836ef3bcc1dd0681d1db9cc7c3a"
server_key_iv: "f7357a72d635f396efbcd95c8d745da2"
server_key_version: "encryptionkey@0"
I20260812 00:40:00.090817 20740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:00.096957 20740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:00.101462 20740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:00.101763 20740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:00.120735 20740 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:41625
I20260812 00:40:00.120765 20860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:41625 every 8 connection(s)
I20260812 00:40:00.122426 20740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:40:00.126742 20863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:00.128171 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20740
I20260812 00:40:00.128377 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:40:00.128961 8145 external_mini_cluster.cc:1428] Setting key 7942d21cc511e637fa42373793e65610
I20260812 00:40:00.137930 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:00.131002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54497 (local address 127.7.244.126:41625)
0812 00:40:00.131970 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:00.131986 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:00.132046 (+ 60us) server_negotiation.cc:407] Connection header received
0812 00:40:00.133957 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:00.133988 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:00.134435 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:00.134832 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:00.136089 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:00.136096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:00.136112 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:40:00.136129 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:00.136200 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:40:00.136226 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:00.136408 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:00.136576 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 00:40:00.136794 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":413,"threads_started":1}
I20260812 00:40:00.142387 20863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e: Bootstrap starting.
I20260812 00:40:00.142638 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:40:00.146930 20863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:00.148396 20863 log.cc:824] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e: Log is configured to *not* fsync() on all Append() calls
I20260812 00:40:00.151145 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:00.148256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58299 (local address 127.7.244.126:41625)
0812 00:40:00.148480 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:00.148485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:00.148827 (+ 342us) server_negotiation.cc:407] Connection header received
0812 00:40:00.149047 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:00.149052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:00.149123 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:00.149261 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:00.150455 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:00.150460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:00.150464 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:40:00.150482 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:00.150528 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:40:00.150545 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:00.150652 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:00.150785 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 00:40:00.150843 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 00:40:00.152333 20863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e: No bootstrap required, opened a new log
I20260812 00:40:00.160379 20863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae5f45170834297adea53788be9048e" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41625 } }
I20260812 00:40:00.160809 20863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:40:00.160926 20863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae5f45170834297adea53788be9048e, State: Initialized, Role: FOLLOWER
I20260812 00:40:00.161583 20863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae5f45170834297adea53788be9048e" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41625 } }
I20260812 00:40:00.161830 20863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:40:00.161984 20863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:40:00.162155 20863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:00.164441 20863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae5f45170834297adea53788be9048e" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41625 } }
I20260812 00:40:00.165004 20863 leader_election.cc:302] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ae5f45170834297adea53788be9048e; no voters:
I20260812 00:40:00.165681 20863 leader_election.cc:288] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:40:00.165846 20883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:40:00.167026 20883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [term 1 LEADER]: Becoming Leader. State: Replica: 1ae5f45170834297adea53788be9048e, State: Running, Role: LEADER
I20260812 00:40:00.167762 20883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae5f45170834297adea53788be9048e" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41625 } }
I20260812 00:40:00.168139 20863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:00.170531 20886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ae5f45170834297adea53788be9048e. Latest consensus state: current_term: 1 leader_uuid: "1ae5f45170834297adea53788be9048e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae5f45170834297adea53788be9048e" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41625 } } }
I20260812 00:40:00.171046 20886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:00.170557 20884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ae5f45170834297adea53788be9048e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae5f45170834297adea53788be9048e" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41625 } } }
I20260812 00:40:00.172372 20884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:00.174486 20897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:40:00.181082 20897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:40:00.190523 20897 catalog_manager.cc:1347] Generated new cluster ID: 9d1525e0600b4ada893c6baea3d679aa
I20260812 00:40:00.190665 20897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:40:00.238759 20897 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:40:00.239251 20897 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:40:00.247201 20897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ae5f45170834297adea53788be9048e: Generated new TSK 0
I20260812 00:40:00.248497 20897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:40:00.263352 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20740
W20260812 00:40:00.264696 20856 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54497
W20260812 00:40:00.265513 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20740: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:40:00.265686 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20740
2026-08-12T00:40:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2294 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:40:00 dist-test-slave-gjw7 krb5kdc[20923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:40:00 dist-test-slave-gjw7 krb5kdc[20923](info): set up 2 sockets
Aug 12 00:40:00 dist-test-slave-gjw7 krb5kdc[20923](info): commencing operation
krb5kdc: starting...
W20260812 00:40:01.606405 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:40:01 dist-test-slave-gjw7 krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:40:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:40:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:40:01.948328 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:36401
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:36941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:36401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:40:02.316692 20959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:02.317036 20959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:02.317184 20959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:02.324770 20959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:02.324925 20959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:02.325006 20959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:02.325088 20959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:02.325151 20959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:02.336973 20959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:36941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:36401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:36401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:02.338513 20959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:02.339996 20959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:02.352934 20968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:02.352898 20969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:02.352964 20971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:02.354423 20959 server_base.cc:1034] running on GCE node
I20260812 00:40:02.355406 20959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:02.357396 20959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:02.358649 20959 hybrid_clock.cc:648] HybridClock initialized: now 1786495202358594 us; error 57 us; skew 500 ppm
I20260812 00:40:02.359079 20959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:02 dist-test-slave-gjw7 krb5kdc[20923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495202, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:02.362758 20959 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:40:02.364656 20959 webserver.cc:460] Webserver started at http://127.7.244.126:33737/ using document root <none> and password file <none>
I20260812 00:40:02.365494 20959 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:02.365629 20959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:02.365954 20959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:02.369432 20959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "f348f4b961914473891b6b9aca7ac5f8"
format_stamp: "Formatted at 2026-08-12 00:40:02 on dist-test-slave-gjw7"
server_key: "90f446cd83ebc2ee3e008a89bca752a7"
server_key_iv: "e8ba412354551d08e826754b297d15d0"
server_key_version: "encryptionkey@0"
I20260812 00:40:02.370354 20959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "f348f4b961914473891b6b9aca7ac5f8"
format_stamp: "Formatted at 2026-08-12 00:40:02 on dist-test-slave-gjw7"
server_key: "90f446cd83ebc2ee3e008a89bca752a7"
server_key_iv: "e8ba412354551d08e826754b297d15d0"
server_key_version: "encryptionkey@0"
I20260812 00:40:02.376681 20959 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260812 00:40:02.380971 20989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:02.382702 20959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:40:02.382929 20959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "f348f4b961914473891b6b9aca7ac5f8"
format_stamp: "Formatted at 2026-08-12 00:40:02 on dist-test-slave-gjw7"
server_key: "90f446cd83ebc2ee3e008a89bca752a7"
server_key_iv: "e8ba412354551d08e826754b297d15d0"
server_key_version: "encryptionkey@0"
I20260812 00:40:02.383157 20959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:02.398567 20959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:02.403396 20959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:02.403729 20959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:02.422685 20959 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:36401
I20260812 00:40:02.422744 21091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:36401 every 8 connection(s)
I20260812 00:40:02.424194 20959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:40:02.428515 21094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:02.433117 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 20959
I20260812 00:40:02.433343 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:40:02.433948 8145 external_mini_cluster.cc:1428] Setting key bade6ce7a9c1e8c4142aa0a3968d788d
I20260812 00:40:02.442529 21094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8: Bootstrap starting.
I20260812 00:40:02.446084 21094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:02.447222 21094 log.cc:824] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:40:02 dist-test-slave-gjw7 krb5kdc[20923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:40:02.450825 21094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8: No bootstrap required, opened a new log
I20260812 00:40:02.459698 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:02.436022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56995 (local address 127.7.244.126:36401)
0812 00:40:02.436893 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:02.436906 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:02.436959 (+ 53us) server_negotiation.cc:407] Connection header received
0812 00:40:02.438716 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:02.438756 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:02.439256 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:02.439684 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:02.441280 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:02.442699 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:02.444070 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:02.444806 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:02.444990 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:02.449664 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:02.449677 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:02.449701 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:02.449759 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:02.453941 (+ 4182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:02.454836 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:02.454852 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:02.454864 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:02.455123 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:02.455653 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:02.455658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:02.455662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:02.456525 (+ 863us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:02.456871 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:02.457563 (+ 692us) server_negotiation.cc:299] Negotiation successful
0812 00:40:02.458104 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":394,"threads_started":1}
I20260812 00:40:02.461622 21094 raft_consensus.cc:348] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f348f4b961914473891b6b9aca7ac5f8" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 36401 } }
I20260812 00:40:02.462149 21094 raft_consensus.cc:374] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:40:02.462281 21094 raft_consensus.cc:729] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f348f4b961914473891b6b9aca7ac5f8, State: Initialized, Role: FOLLOWER
I20260812 00:40:02.462986 21094 consensus_queue.cc:260] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f348f4b961914473891b6b9aca7ac5f8" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 36401 } }
I20260812 00:40:02.463289 21094 raft_consensus.cc:388] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:40:02.463442 21094 raft_consensus.cc:482] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:40:02.463639 21094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:02.464589 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:40:02.467028 21094 raft_consensus.cc:504] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f348f4b961914473891b6b9aca7ac5f8" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 36401 } }
I20260812 00:40:02.467770 21094 leader_election.cc:302] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f348f4b961914473891b6b9aca7ac5f8; no voters:
I20260812 00:40:02.468545 21094 leader_election.cc:288] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:40:02.468720 21105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:40:02.469460 21105 raft_consensus.cc:686] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [term 1 LEADER]: Becoming Leader. State: Replica: f348f4b961914473891b6b9aca7ac5f8, State: Running, Role: LEADER
I20260812 00:40:02.470220 21105 consensus_queue.cc:237] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f348f4b961914473891b6b9aca7ac5f8" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 36401 } }
I20260812 00:40:02.471956 21094 sys_catalog.cc:564] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:02.473631 21109 sys_catalog.cc:455] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f348f4b961914473891b6b9aca7ac5f8. Latest consensus state: current_term: 1 leader_uuid: "f348f4b961914473891b6b9aca7ac5f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f348f4b961914473891b6b9aca7ac5f8" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 36401 } } }
I20260812 00:40:02.474248 21113 sys_catalog.cc:455] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f348f4b961914473891b6b9aca7ac5f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f348f4b961914473891b6b9aca7ac5f8" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 36401 } } }
I20260812 00:40:02.474644 21113 sys_catalog.cc:458] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:02.475101 21109 sys_catalog.cc:458] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:02.485956 21120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:40:02.489656 21097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:02.470530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32831 (local address 127.7.244.126:36401)
0812 00:40:02.470795 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:02.470801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:02.471452 (+ 651us) server_negotiation.cc:407] Connection header received
0812 00:40:02.471661 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:02.471666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:02.471750 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:02.471859 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:02.473952 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:02.475103 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:02.477152 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:02.477918 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:02.478036 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:02.481762 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:02.481771 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:02.481776 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:02.481832 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:02.486046 (+ 4214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:02.487090 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:02.487097 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:02.487101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:02.487301 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:02.487760 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:02.487766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:02.487768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:02.488198 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:02.488472 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:02.489019 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 00:40:02.489247 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 00:40:02.495427 21120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:40:02.506542 21120 catalog_manager.cc:1347] Generated new cluster ID: aaa17c5fc74842b383815a13f51f6b2d
I20260812 00:40:02.506700 21120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:40:02.518765 21120 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:40:02.520828 21120 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:40:02.539814 21120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f348f4b961914473891b6b9aca7ac5f8: Generated new TSK 0
I20260812 00:40:02.541076 21120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:40:02.549912 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 20959
I20260812 00:40:02.551388 21078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:40:02.643713 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 20959
2026-08-12T00:40:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:40:02 dist-test-slave-gjw7 krb5kdc[21149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:40:02 dist-test-slave-gjw7 krb5kdc[21149](info): set up 2 sockets
Aug 12 00:40:02 dist-test-slave-gjw7 krb5kdc[21149](info): commencing operation
krb5kdc: starting...
W20260812 00:40:04.033497 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:40:04 dist-test-slave-gjw7 krb5kdc[21149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:40:04.368822 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:37157
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:43563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:37157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:40:04.733244 21169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:04.733603 21169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:04.733741 21169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:04.741274 21169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:04.741439 21169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:04.741513 21169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:04.741600 21169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:04.741657 21169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:04.752249 21169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:43563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:37157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:37157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:04.753696 21169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:04.755092 21169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:04.767582 21177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:04.767545 21181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:04.767505 21176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:04.768296 21169 server_base.cc:1034] running on GCE node
I20260812 00:40:04.769223 21169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:04.770805 21169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:04.772023 21169 hybrid_clock.cc:648] HybridClock initialized: now 1786495204771975 us; error 40 us; skew 500 ppm
I20260812 00:40:04.772384 21169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:04 dist-test-slave-gjw7 krb5kdc[21149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495204, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:04.775686 21169 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:40:04.777354 21169 webserver.cc:460] Webserver started at http://127.7.244.126:39429/ using document root <none> and password file <none>
I20260812 00:40:04.778129 21169 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:04.778257 21169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:04.778573 21169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:04.781858 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "a8f751febcb7449094c330e30b761ad0"
format_stamp: "Formatted at 2026-08-12 00:40:04 on dist-test-slave-gjw7"
server_key: "bfb21fa5c54497c8c7b2a707d3f6a0c8"
server_key_iv: "8c06e39d8a9b88c45285a847b726fb0e"
server_key_version: "encryptionkey@0"
I20260812 00:40:04.782693 21169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "a8f751febcb7449094c330e30b761ad0"
format_stamp: "Formatted at 2026-08-12 00:40:04 on dist-test-slave-gjw7"
server_key: "bfb21fa5c54497c8c7b2a707d3f6a0c8"
server_key_iv: "8c06e39d8a9b88c45285a847b726fb0e"
server_key_version: "encryptionkey@0"
I20260812 00:40:04.788507 21169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260812 00:40:04.792562 21194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:04.794368 21169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 00:40:04.794631 21169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "a8f751febcb7449094c330e30b761ad0"
format_stamp: "Formatted at 2026-08-12 00:40:04 on dist-test-slave-gjw7"
server_key: "bfb21fa5c54497c8c7b2a707d3f6a0c8"
server_key_iv: "8c06e39d8a9b88c45285a847b726fb0e"
server_key_version: "encryptionkey@0"
I20260812 00:40:04.794859 21169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:04.816151 21169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:04.820703 21169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:04.820991 21169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:04.839172 21169 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:37157
I20260812 00:40:04.839226 21290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:37157 every 8 connection(s)
I20260812 00:40:04.840690 21169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:40:04.842870 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21169
I20260812 00:40:04.843083 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:40:04.843662 8145 external_mini_cluster.cc:1428] Setting key 9598358fef6ebde2ed988d2df9dc8ae2
I20260812 00:40:04.845757 21291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:04.860034 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:04.845661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52445 (local address 127.7.244.126:37157)
0812 00:40:04.846553 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:04.846568 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:04.846620 (+ 52us) server_negotiation.cc:407] Connection header received
0812 00:40:04.848934 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:04.848969 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:04.849625 (+ 656us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:04.850140 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:04.850714 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:04.852794 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:04.854270 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:04.855402 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:04.855606 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:04.856867 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:04.856883 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:04.856904 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:40:04.857726 (+ 822us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:04.857845 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:40:04.857881 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:04.858017 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:04.858235 (+ 218us) server_negotiation.cc:299] Negotiation successful
0812 00:40:04.858586 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":374,"threads_started":1}
I20260812 00:40:04.861332 21291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0: Bootstrap starting.
I20260812 00:40:04.864502 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:40:04.865432 21291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:04.866809 21291 log.cc:824] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0: Log is configured to *not* fsync() on all Append() calls
I20260812 00:40:04.870306 21291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0: No bootstrap required, opened a new log
I20260812 00:40:04.877331 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:04.869864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48425 (local address 127.7.244.126:37157)
0812 00:40:04.870100 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:04.870107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:04.870541 (+ 434us) server_negotiation.cc:407] Connection header received
0812 00:40:04.870726 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:04.870730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:04.870795 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:04.870900 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:04.871564 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:04.872292 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:04.874199 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:04.874828 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:04.874986 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:04.876383 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:04.876390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:04.876395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:40:04.876425 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:04.876492 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:40:04.876515 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:04.876607 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:04.876805 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 00:40:04.876923 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:40:04.880810 21291 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f751febcb7449094c330e30b761ad0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37157 } }
I20260812 00:40:04.881371 21291 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:40:04.881515 21291 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8f751febcb7449094c330e30b761ad0, State: Initialized, Role: FOLLOWER
I20260812 00:40:04.882126 21291 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f751febcb7449094c330e30b761ad0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37157 } }
I20260812 00:40:04.882352 21291 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:40:04.882463 21291 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:40:04.882602 21291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:04.884812 21291 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f751febcb7449094c330e30b761ad0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37157 } }
I20260812 00:40:04.885380 21291 leader_election.cc:302] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8f751febcb7449094c330e30b761ad0; no voters:
I20260812 00:40:04.885986 21291 leader_election.cc:288] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:40:04.886133 21310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:40:04.886965 21310 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [term 1 LEADER]: Becoming Leader. State: Replica: a8f751febcb7449094c330e30b761ad0, State: Running, Role: LEADER
I20260812 00:40:04.887513 21310 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f751febcb7449094c330e30b761ad0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37157 } }
I20260812 00:40:04.888132 21291 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:04.890717 21311 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8f751febcb7449094c330e30b761ad0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f751febcb7449094c330e30b761ad0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37157 } } }
I20260812 00:40:04.890587 21312 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8f751febcb7449094c330e30b761ad0. Latest consensus state: current_term: 1 leader_uuid: "a8f751febcb7449094c330e30b761ad0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8f751febcb7449094c330e30b761ad0" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 37157 } } }
I20260812 00:40:04.891141 21311 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:04.891150 21312 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:04.892792 21329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:40:04.898021 21329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:40:04.907157 21329 catalog_manager.cc:1347] Generated new cluster ID: 75f28a57be1b42debc74908f948af96e
I20260812 00:40:04.907302 21329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:40:04.917240 21329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:40:04.918722 21329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:40:04.927644 21329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8f751febcb7449094c330e30b761ad0: Generated new TSK 0
I20260812 00:40:04.928588 21329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:40:04.938194 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21169
W20260812 00:40:04.939472 21279 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52445
W20260812 00:40:04.940225 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21169: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:40:04.940394 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21169
2026-08-12T00:40:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:40:05 dist-test-slave-gjw7 krb5kdc[21352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:40:05 dist-test-slave-gjw7 krb5kdc[21352](info): set up 2 sockets
Aug 12 00:40:05 dist-test-slave-gjw7 krb5kdc[21352](info): commencing operation
krb5kdc: starting...
W20260812 00:40:06.286368 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:40:06 dist-test-slave-gjw7 krb5kdc[21352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:40:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:40:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:40:06.622869 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:38947
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:38947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:40:06.983578 21386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:06.983933 21386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:06.984069 21386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:06.991577 21386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:06.991710 21386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:06.991771 21386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:06.991859 21386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:06.991935 21386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:07.002329 21386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:38947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:38947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:07.003825 21386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:07.005280 21386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:07.018285 21404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:07.018329 21397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:07.018357 21396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:07.019210 21386 server_base.cc:1034] running on GCE node
I20260812 00:40:07.020099 21386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:07.021857 21386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:07.023114 21386 hybrid_clock.cc:648] HybridClock initialized: now 1786495207023051 us; error 56 us; skew 500 ppm
I20260812 00:40:07.023535 21386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:07 dist-test-slave-gjw7 krb5kdc[21352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495207, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:07.027083 21386 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:40:07.028894 21386 webserver.cc:460] Webserver started at http://127.7.244.126:33419/ using document root <none> and password file <none>
I20260812 00:40:07.029774 21386 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:07.029922 21386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:07.030258 21386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:07.033690 21386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "470c612118da425abce87a19b0fd0801"
format_stamp: "Formatted at 2026-08-12 00:40:07 on dist-test-slave-gjw7"
server_key: "de8fe50a798b98494701c141e51e14a3"
server_key_iv: "e86b0a222fe9fe7b0c48ee7db7a4707c"
server_key_version: "encryptionkey@0"
I20260812 00:40:07.034576 21386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "470c612118da425abce87a19b0fd0801"
format_stamp: "Formatted at 2026-08-12 00:40:07 on dist-test-slave-gjw7"
server_key: "de8fe50a798b98494701c141e51e14a3"
server_key_iv: "e86b0a222fe9fe7b0c48ee7db7a4707c"
server_key_version: "encryptionkey@0"
I20260812 00:40:07.040680 21386 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260812 00:40:07.044842 21414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:07.046582 21386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 00:40:07.046792 21386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "470c612118da425abce87a19b0fd0801"
format_stamp: "Formatted at 2026-08-12 00:40:07 on dist-test-slave-gjw7"
server_key: "de8fe50a798b98494701c141e51e14a3"
server_key_iv: "e86b0a222fe9fe7b0c48ee7db7a4707c"
server_key_version: "encryptionkey@0"
I20260812 00:40:07.047034 21386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:07.053972 21386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:07.058956 21386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:07.059303 21386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:07.078115 21386 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:38947
I20260812 00:40:07.078158 21504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:38947 every 8 connection(s)
I20260812 00:40:07.079623 21386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:40:07.083592 21505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:07.086444 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21386
I20260812 00:40:07.086632 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:40:07.087159 8145 external_mini_cluster.cc:1428] Setting key f4a5cf2053a1b2636d2beb6bcf343e89
I20260812 00:40:07.095211 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:07.089065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53307 (local address 127.7.244.126:38947)
0812 00:40:07.089931 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:07.089943 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:07.089988 (+ 45us) server_negotiation.cc:407] Connection header received
0812 00:40:07.091655 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:07.091680 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:07.092057 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:07.092415 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:07.093529 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:07.093535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:07.093553 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:40:07.093572 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:07.093641 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:40:07.093666 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:07.093850 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:07.093981 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 00:40:07.094193 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":396,"threads_started":1}
I20260812 00:40:07.096160 21505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801: Bootstrap starting.
I20260812 00:40:07.099121 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:40:07.099637 21505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:07.100926 21505 log.cc:824] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801: Log is configured to *not* fsync() on all Append() calls
I20260812 00:40:07.104234 21505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801: No bootstrap required, opened a new log
I20260812 00:40:07.106976 21512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:07.104157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60457 (local address 127.7.244.126:38947)
0812 00:40:07.104368 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:07.104376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:07.104888 (+ 512us) server_negotiation.cc:407] Connection header received
0812 00:40:07.105075 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:07.105079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:07.105130 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:07.105211 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:07.106294 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:07.106298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:07.106302 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 00:40:07.106322 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:07.106366 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 00:40:07.106386 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:07.106490 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:07.106609 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 00:40:07.106666 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 00:40:07.112929 21505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470c612118da425abce87a19b0fd0801" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38947 } }
I20260812 00:40:07.113464 21505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 00:40:07.113585 21505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470c612118da425abce87a19b0fd0801, State: Initialized, Role: FOLLOWER
I20260812 00:40:07.114279 21505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470c612118da425abce87a19b0fd0801" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38947 } }
I20260812 00:40:07.114560 21505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 00:40:07.114706 21505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 00:40:07.114874 21505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:07.117199 21505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470c612118da425abce87a19b0fd0801" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38947 } }
I20260812 00:40:07.117799 21505 leader_election.cc:302] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 470c612118da425abce87a19b0fd0801; no voters:
I20260812 00:40:07.118440 21505 leader_election.cc:288] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 00:40:07.118595 21523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:40:07.119292 21523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [term 1 LEADER]: Becoming Leader. State: Replica: 470c612118da425abce87a19b0fd0801, State: Running, Role: LEADER
I20260812 00:40:07.119937 21523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470c612118da425abce87a19b0fd0801" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38947 } }
I20260812 00:40:07.120801 21505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:07.123257 21528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 470c612118da425abce87a19b0fd0801. Latest consensus state: current_term: 1 leader_uuid: "470c612118da425abce87a19b0fd0801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470c612118da425abce87a19b0fd0801" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38947 } } }
I20260812 00:40:07.123140 21524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "470c612118da425abce87a19b0fd0801" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470c612118da425abce87a19b0fd0801" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 38947 } } }
I20260812 00:40:07.123685 21528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:07.123781 21524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:07.125586 21541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:40:07.130807 21541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:40:07.140793 21541 catalog_manager.cc:1347] Generated new cluster ID: 35c607334cba48d98e3cd185102527c3
I20260812 00:40:07.140928 21541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:40:07.150701 21541 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:40:07.151221 21541 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:40:07.160590 21541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 470c612118da425abce87a19b0fd0801: Generated new TSK 0
I20260812 00:40:07.161607 21541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:40:07.173019 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21386
W20260812 00:40:07.174459 21490 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53307
W20260812 00:40:07.175344 8145 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21386: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 00:40:07.175518 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21386
2026-08-12T00:40:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2242 ms)
[----------] 18 tests from AuthTokenIssuingTest (41685 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:40:07 dist-test-slave-gjw7 krb5kdc[21559](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:40:07 dist-test-slave-gjw7 krb5kdc[21559](info): set up 2 sockets
Aug 12 00:40:07 dist-test-slave-gjw7 krb5kdc[21559](info): commencing operation
krb5kdc: starting...
W20260812 00:40:08.544718 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:40:08 dist-test-slave-gjw7 krb5kdc[21559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:40:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:40:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:40:08.885730 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41181
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:41181,127.7.244.125:35939,127.7.244.124:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:40:09.257757 21590 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:40:09.258127 21590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:09.258240 21590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:09.258356 21590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:09.266103 21590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:09.266227 21590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:09.266287 21590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:09.266366 21590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:09.266456 21590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:09.276844 21590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:41181,127.7.244.125:35939,127.7.244.124:33461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:09.278270 21590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:09.279639 21590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:09.292042 21597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:09.292055 21601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:09.291932 21599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:09.293104 21590 server_base.cc:1034] running on GCE node
I20260812 00:40:09.294005 21590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:09.295567 21590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:09.296819 21590 hybrid_clock.cc:648] HybridClock initialized: now 1786495209296741 us; error 42 us; skew 500 ppm
I20260812 00:40:09.297189 21590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:09 dist-test-slave-gjw7 krb5kdc[21559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:09.300796 21590 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:40:09.302543 21590 webserver.cc:460] Webserver started at http://127.7.244.126:42995/ using document root <none> and password file <none>
I20260812 00:40:09.303321 21590 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:09.303434 21590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:09.303776 21590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:09.307204 21590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "c928c479d5bc497a8c7a72e545255597"
format_stamp: "Formatted at 2026-08-12 00:40:09 on dist-test-slave-gjw7"
server_key: "1e539094b88a7ffeb2196c6965e7286c"
server_key_iv: "8cd325a3b2274e1e6a0b672d45601f15"
server_key_version: "encryptionkey@0"
I20260812 00:40:09.308109 21590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "c928c479d5bc497a8c7a72e545255597"
format_stamp: "Formatted at 2026-08-12 00:40:09 on dist-test-slave-gjw7"
server_key: "1e539094b88a7ffeb2196c6965e7286c"
server_key_iv: "8cd325a3b2274e1e6a0b672d45601f15"
server_key_version: "encryptionkey@0"
I20260812 00:40:09.314426 21590 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260812 00:40:09.318341 21612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:09.319842 21590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:40:09.320024 21590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "c928c479d5bc497a8c7a72e545255597"
format_stamp: "Formatted at 2026-08-12 00:40:09 on dist-test-slave-gjw7"
server_key: "1e539094b88a7ffeb2196c6965e7286c"
server_key_iv: "8cd325a3b2274e1e6a0b672d45601f15"
server_key_version: "encryptionkey@0"
I20260812 00:40:09.320259 21590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:09.340693 21590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:09.345152 21590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:09.345485 21590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:09.364779 21590 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:41181
I20260812 00:40:09.364792 21707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:41181 every 8 connection(s)
I20260812 00:40:09.366287 21590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:40:09.370460 21708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:09.371174 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21590
I20260812 00:40:09.371402 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:40:09.372262 8145 external_mini_cluster.cc:1428] Setting key 3479babe92a055d4983346434fcd0246
I20260812 00:40:09.379976 21708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:40:09 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:40:09.406244 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.383926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41181 (local address 127.0.0.1:33272)
0812 00:40:09.384795 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.384853 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:09.386991 (+ 2138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:09.387935 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:09.387941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:09.388738 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:09.389229 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:09.389256 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.391064 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.391069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:09.392735 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:09.392753 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.393867 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.393882 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:09.394070 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:09.394834 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:09.394863 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:09.397753 (+ 2890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:09.401406 (+ 3653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:09.401421 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:09.401436 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:09.401963 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:09.402607 (+ 644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:09.402611 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:09.402614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:09.402816 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:09.404014 (+ 1198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:09.404021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:09.404254 (+ 233us) client_negotiation.cc:769] Sending connection context
0812 00:40:09.404501 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 00:40:09.404986 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":374,"threads_started":1}
I20260812 00:40:09.406284 21713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.385278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33272 (local address 127.7.244.126:41181)
0812 00:40:09.385997 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:09.386006 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:09.386061 (+ 55us) server_negotiation.cc:407] Connection header received
0812 00:40:09.387404 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:09.387410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:09.387672 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:09.387869 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:09.389550 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.390847 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.392921 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.393601 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.393764 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:09.398037 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:09.398047 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:09.398063 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:09.398102 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:09.401210 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:09.402193 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:09.402204 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:09.402208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:09.402459 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:09.402981 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:09.402987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:09.402990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:09.403818 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:09.404085 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:09.404716 (+ 631us) server_negotiation.cc:299] Negotiation successful
0812 00:40:09.405205 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":519,"thread_start_us":331,"threads_started":1}
WARNING: no policy specified for kudu/127.7.244.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.125@KRBTEST.COM" created.
I20260812 00:40:09.409919 21708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:09.411240 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.410736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:52550)
0812 00:40:09.410939 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.411082 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260812 00:40:09.411554 21616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
I20260812 00:40:09.413205 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.412787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:52556)
0812 00:40:09.412933 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.413034 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 00:40:09.413455 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:35939: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab.
Entry for principal kudu/127.7.244.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab.
I20260812 00:40:09.448324 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.125:35939
--webserver_interface=127.7.244.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab
--principal=kudu/127.7.244.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:41181,127.7.244.125:35939,127.7.244.124:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:40:09.462862 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } attempt: 1
I20260812 00:40:09.464270 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.463757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:52564)
0812 00:40:09.463966 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.464084 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 00:40:09.465461 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.465041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:52580)
0812 00:40:09.465182 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.465269 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 00:40:09.465660 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:35939: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
I20260812 00:40:09.533963 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } attempt: 2
I20260812 00:40:09.535310 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.534790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:52584)
0812 00:40:09.535049 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.535149 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 00:40:09.536435 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.536028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:52586)
0812 00:40:09.536189 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.536287 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 00:40:09.536654 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:35939: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
I20260812 00:40:09.627916 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } attempt: 3
I20260812 00:40:09.629134 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.628678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:52598)
0812 00:40:09.628898 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.628996 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 00:40:09.630211 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.629837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:52602)
0812 00:40:09.629992 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.630066 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 00:40:09.630417 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:35939: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
I20260812 00:40:09.773694 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } attempt: 4
I20260812 00:40:09.775110 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.774467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:60984)
0812 00:40:09.774859 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.774962 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260812 00:40:09.776248 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.775843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:60992)
0812 00:40:09.776016 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.776108 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 00:40:09.776472 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:35939: Network error: Client connection negotiation failed: client connection to 127.7.244.125:35939: connect: Connection refused (error 111)
W20260812 00:40:09.807880 21718 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:40:09.808308 21718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:09.808413 21718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:09.808528 21718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:09.816355 21718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:09.816483 21718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:09.816565 21718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:09.816630 21718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:09.816720 21718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:09.827411 21718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:41181,127.7.244.125:35939,127.7.244.124:33461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab
--principal=kudu/127.7.244.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.125:35939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.244.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:09.828860 21718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:09.830379 21718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:09.842861 21727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:09.842988 21725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:09.842887 21732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:09.843690 21718 server_base.cc:1034] running on GCE node
I20260812 00:40:09.844518 21718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:09.846081 21718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:09.847333 21718 hybrid_clock.cc:648] HybridClock initialized: now 1786495209847259 us; error 56 us; skew 500 ppm
I20260812 00:40:09.847694 21718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:09 dist-test-slave-gjw7 krb5kdc[21559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:09.851291 21718 init.cc:377] Logged in from keytab as kudu/127.7.244.125@KRBTEST.COM (short username kudu)
I20260812 00:40:09.852974 21718 webserver.cc:460] Webserver started at http://127.7.244.125:36369/ using document root <none> and password file <none>
I20260812 00:40:09.853787 21718 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:09.853917 21718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:09.854259 21718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:09.857648 21718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/data/instance:
uuid: "5206af2599a542d59edfb89570bbc458"
format_stamp: "Formatted at 2026-08-12 00:40:09 on dist-test-slave-gjw7"
server_key: "ec34e0587f02ba67e0e32cd5aa475732"
server_key_iv: "5b4a50401eb25eddc584c3b96511b84a"
server_key_version: "encryptionkey@0"
I20260812 00:40:09.858557 21718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/wal/instance:
uuid: "5206af2599a542d59edfb89570bbc458"
format_stamp: "Formatted at 2026-08-12 00:40:09 on dist-test-slave-gjw7"
server_key: "ec34e0587f02ba67e0e32cd5aa475732"
server_key_iv: "5b4a50401eb25eddc584c3b96511b84a"
server_key_version: "encryptionkey@0"
I20260812 00:40:09.864444 21718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 00:40:09.868337 21747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:09.870157 21718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:40:09.870396 21718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/wal
uuid: "5206af2599a542d59edfb89570bbc458"
format_stamp: "Formatted at 2026-08-12 00:40:09 on dist-test-slave-gjw7"
server_key: "ec34e0587f02ba67e0e32cd5aa475732"
server_key_iv: "5b4a50401eb25eddc584c3b96511b84a"
server_key_version: "encryptionkey@0"
I20260812 00:40:09.870651 21718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:09.886116 21718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:09.890622 21718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:09.890874 21718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:09.909701 21718 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.125:35939
I20260812 00:40:09.909720 21857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.125:35939 every 8 connection(s)
I20260812 00:40:09.911228 21718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
I20260812 00:40:09.912694 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21718
I20260812 00:40:09.912889 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-1/wal/instance
I20260812 00:40:09.913494 8145 external_mini_cluster.cc:1428] Setting key c61eca725528904dcac906ff806d7d18
I20260812 00:40:09.916508 21859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:09.926373 21859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:40:09 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.244.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.124@KRBTEST.COM" created.
I20260812 00:40:09.949626 21864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.930016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.7.244.126:41181)
0812 00:40:09.930682 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:09.930687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:09.931918 (+ 1231us) server_negotiation.cc:407] Connection header received
0812 00:40:09.934127 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:09.934133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:09.934239 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:09.934342 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:09.936203 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.937023 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.938739 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.939171 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.939295 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:09.943157 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:09.943163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:09.943166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:09.943209 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:09.946172 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:09.947082 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:09.947087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:09.947089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:09.947228 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:09.947698 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:09.947702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:09.947704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:09.948017 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:09.948160 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:09.948952 (+ 792us) server_negotiation.cc:299] Negotiation successful
0812 00:40:09.949204 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":422,"threads_started":1}
I20260812 00:40:09.950731 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.930832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41181 (local address 127.0.0.1:35496)
0812 00:40:09.931725 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.931768 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:09.933771 (+ 2003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:09.934653 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:09.934664 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:09.935268 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:09.935948 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:09.935972 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.937212 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.937217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:09.938559 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:09.938575 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.939438 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.939446 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:09.939698 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:09.940435 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:09.940461 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:09.942866 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:09.946369 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:09.946379 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:09.946396 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:09.946927 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:09.947355 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:09.947361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:09.947364 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:09.947568 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:09.948166 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:09.948176 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:09.948563 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 00:40:09.948818 (+ 255us) client_negotiation.cc:240] Negotiation successful
0812 00:40:09.949457 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":387,"threads_started":1}
I20260812 00:40:09.953145 21859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:40:09 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:40:09.969842 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.954097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:32770)
0812 00:40:09.954301 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.954326 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:09.954497 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:09.955610 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:09.955614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:09.956056 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:09.956210 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:09.956219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.958067 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.958072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:09.958950 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:09.958963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.960102 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.960107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:09.960264 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:09.960901 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:09.960918 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:09.963251 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:09.966810 (+ 3559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:09.966816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:09.966820 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:09.967414 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:09.967849 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:09.967854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:09.967858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:09.968046 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:09.968729 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:09.968734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:09.968888 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 00:40:09.969032 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 00:40:09.969401 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 00:40:09.969878 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.954251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.7.244.125:35939)
0812 00:40:09.955034 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:09.955043 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:09.955077 (+ 34us) server_negotiation.cc:407] Connection header received
0812 00:40:09.955143 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:09.955150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:09.955442 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:09.955608 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:09.956384 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.957931 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.959100 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:09.959938 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:09.960063 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:09.963529 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:09.963546 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:09.963564 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:09.963599 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:09.966621 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:09.967579 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:09.967587 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:09.967589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:09.967729 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:09.968169 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:09.968173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:09.968175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:09.968581 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:09.968755 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:09.969190 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 00:40:09.969537 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":417,"threads_started":1}
I20260812 00:40:09.972756 21859 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:09.974032 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.973473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51584)
0812 00:40:09.973672 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.973815 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 00:40:09.974335 21755 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
I20260812 00:40:09.976135 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:09.975676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51592)
0812 00:40:09.975875 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:09.975994 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 00:40:09.976387 21859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab.
Entry for principal kudu/127.7.244.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab.
I20260812 00:40:09.992538 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.124:33461
--webserver_interface=127.7.244.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab
--principal=kudu/127.7.244.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:41181,127.7.244.125:35939,127.7.244.124:33461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:40:10.025765 21859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 1
I20260812 00:40:10.027267 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.026659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51604)
0812 00:40:10.026913 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.027063 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 00:40:10.028478 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.028034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51612)
0812 00:40:10.028210 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.028317 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 00:40:10.028741 21859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
I20260812 00:40:10.075723 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } attempt: 5
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:40:10.091687 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.076671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.7.244.125:35939)
0812 00:40:10.076860 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.076864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.076885 (+ 21us) server_negotiation.cc:407] Connection header received
0812 00:40:10.077131 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.077136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.077232 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.077380 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.078434 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.079144 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.080899 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.081460 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.081574 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.085700 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.085706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.085710 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.085763 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.088495 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.089519 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.089524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.089527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.089680 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.090144 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.090152 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.090154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.090496 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.090618 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.091239 (+ 621us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.091462 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260812 00:40:10.091773 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.076534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:32772)
0812 00:40:10.076768 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.076793 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:10.076939 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:10.077435 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:10.077440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:10.078003 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:10.078250 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.078263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.079290 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.079296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.080636 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.080666 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.081638 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.081644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.081880 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.082569 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:10.082594 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:10.085421 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:10.088678 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.088684 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.088688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.089358 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.089819 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.089824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.089828 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.090035 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.090633 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:10.090638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:10.090844 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 00:40:10.091059 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 00:40:10.091423 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 00:40:10.092960 21708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:10.094203 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.093712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51624)
0812 00:40:10.093938 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.094031 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260812 00:40:10.095225 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.094889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51626)
0812 00:40:10.095016 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.095111 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 00:40:10.095438 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
I20260812 00:40:10.096978 21859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 2
I20260812 00:40:10.098202 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.097665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51642)
0812 00:40:10.097877 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.098012 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 00:40:10.099226 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.098885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51650)
0812 00:40:10.099016 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.099092 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 00:40:10.099529 21859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
I20260812 00:40:10.146703 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 1
I20260812 00:40:10.147898 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.147408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51654)
0812 00:40:10.147619 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.147713 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 00:40:10.149009 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.148686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51658)
0812 00:40:10.148821 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.148896 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 00:40:10.149230 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
I20260812 00:40:10.190798 21859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 3
I20260812 00:40:10.192065 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.191549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51668)
0812 00:40:10.191773 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.191888 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 00:40:10.193405 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.192831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51684)
0812 00:40:10.193083 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.193204 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260812 00:40:10.193638 21859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
I20260812 00:40:10.217496 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 2
I20260812 00:40:10.218954 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.218340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51696)
0812 00:40:10.218616 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.218729 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 00:40:10.220049 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.219716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51698)
0812 00:40:10.219853 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.219932 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 00:40:10.220296 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
I20260812 00:40:10.326514 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 3
I20260812 00:40:10.327646 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.327152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51712)
0812 00:40:10.327384 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.327474 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 00:40:10.328946 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.328488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51728)
0812 00:40:10.328711 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.328799 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260812 00:40:10.329229 21708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
I20260812 00:40:10.336879 21859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 4
I20260812 00:40:10.338263 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.337733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51736)
0812 00:40:10.337984 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.338084 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 00:40:10.339506 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.339072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51738)
0812 00:40:10.339260 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.339352 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 00:40:10.339896 21859 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:33461: Network error: Client connection negotiation failed: client connection to 127.7.244.124:33461: connect: Connection refused (error 111)
W20260812 00:40:10.369632 21872 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:40:10.370151 21872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:10.370262 21872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:10.370388 21872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:10.380097 21872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:10.380214 21872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:10.380272 21872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:10.380323 21872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:10.380394 21872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:10.390790 21872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:41181,127.7.244.125:35939,127.7.244.124:33461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab
--principal=kudu/127.7.244.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.124:33461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.244.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:10.392201 21872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:10.393785 21872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:10.407725 21886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:10.407644 21890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:10.407779 21885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:10.408792 21872 server_base.cc:1034] running on GCE node
I20260812 00:40:10.409840 21872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:10.411414 21872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:10.412676 21872 hybrid_clock.cc:648] HybridClock initialized: now 1786495210412593 us; error 59 us; skew 500 ppm
I20260812 00:40:10.413060 21872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495210, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:10.416585 21872 init.cc:377] Logged in from keytab as kudu/127.7.244.124@KRBTEST.COM (short username kudu)
I20260812 00:40:10.418311 21872 webserver.cc:460] Webserver started at http://127.7.244.124:45493/ using document root <none> and password file <none>
I20260812 00:40:10.419078 21872 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:10.419193 21872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:10.419575 21872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:10.422876 21872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/data/instance:
uuid: "75deb06bffdb4273863d49db34f1b577"
format_stamp: "Formatted at 2026-08-12 00:40:10 on dist-test-slave-gjw7"
server_key: "eb1c8178ad7cb615941d5241bbe9fa3d"
server_key_iv: "a6bd9da2597b853db532a24bb3caa6a9"
server_key_version: "encryptionkey@0"
I20260812 00:40:10.423827 21872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/wal/instance:
uuid: "75deb06bffdb4273863d49db34f1b577"
format_stamp: "Formatted at 2026-08-12 00:40:10 on dist-test-slave-gjw7"
server_key: "eb1c8178ad7cb615941d5241bbe9fa3d"
server_key_iv: "a6bd9da2597b853db532a24bb3caa6a9"
server_key_version: "encryptionkey@0"
I20260812 00:40:10.429877 21872 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:40:10.433815 21900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:10.435389 21872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 00:40:10.435593 21872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/wal
uuid: "75deb06bffdb4273863d49db34f1b577"
format_stamp: "Formatted at 2026-08-12 00:40:10 on dist-test-slave-gjw7"
server_key: "eb1c8178ad7cb615941d5241bbe9fa3d"
server_key_iv: "a6bd9da2597b853db532a24bb3caa6a9"
server_key_version: "encryptionkey@0"
I20260812 00:40:10.435827 21872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:10.469065 21872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:10.473747 21872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:10.474026 21872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:10.492419 21872 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.124:33461
I20260812 00:40:10.492451 21999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.124:33461 every 8 connection(s)
I20260812 00:40:10.494005 21872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
I20260812 00:40:10.498427 22003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:10.499711 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 21872
I20260812 00:40:10.499924 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786494824837501-8145-0/minicluster-data/master-2/wal/instance
I20260812 00:40:10.500509 8145 external_mini_cluster.cc:1428] Setting key c136ab5287569c3fbe37786b91c3d017
I20260812 00:40:10.506523 21708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 4
I20260812 00:40:10.509204 22003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495210, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:40:10.526607 21712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.507241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51742)
0812 00:40:10.507538 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.507559 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:10.507671 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:10.508614 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:10.508619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:10.509076 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:10.509237 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.509248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.511059 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.511066 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.512512 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.512534 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.514227 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.514234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.514462 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.515165 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:10.515185 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:10.517775 (+ 2590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:10.523036 (+ 5261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.523042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.523047 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.523630 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.524088 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.524094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.524098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.524302 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.525487 (+ 1185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:10.525492 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:10.525701 (+ 209us) client_negotiation.cc:769] Sending connection context
0812 00:40:10.525843 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 00:40:10.526169 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173}
I20260812 00:40:10.528047 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.502757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51740 (local address 127.7.244.124:33461)
0812 00:40:10.503665 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.503680 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.503730 (+ 50us) server_negotiation.cc:407] Connection header received
0812 00:40:10.505589 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.505620 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.506142 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.506555 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.508889 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.510883 (+ 1994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.512772 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.514507 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.514649 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.519695 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.519702 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.519706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.519746 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.523267 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.524263 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.524268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.524271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.524405 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.525004 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.525042 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.525045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.525483 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.525781 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.526345 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.526857 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1245,"server-negotiator.queue_time_us":664,"thread_start_us":398,"threads_started":1}
I20260812 00:40:10.528084 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.507447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.7.244.124:33461)
0812 00:40:10.508219 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.508225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.508249 (+ 24us) server_negotiation.cc:407] Connection header received
0812 00:40:10.508308 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.508312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.508399 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.508511 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.509463 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.510851 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.512725 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.513999 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.514261 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.518044 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.518054 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.518067 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.518120 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.522777 (+ 4657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.523762 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.523772 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.523782 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.523950 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.524493 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.524497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.524500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.525277 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.527148 (+ 1871us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.527277 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.527503 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":619,"thread_start_us":417,"threads_started":1}
I20260812 00:40:10.531165 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.512446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:41181 (local address 127.0.0.1:35500)
0812 00:40:10.513526 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.513576 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:10.513730 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:10.514843 (+ 1113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:10.514852 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:10.515725 (+ 873us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:10.516110 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.516124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.517259 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.517263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.518731 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.518747 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.519621 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.519629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.519845 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.520563 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:10.520589 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:10.523631 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:10.527911 (+ 4280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.527922 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.527929 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.528511 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.528902 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.528907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.528910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.529084 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.529967 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:10.529975 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:10.530172 (+ 197us) client_negotiation.cc:769] Sending connection context
0812 00:40:10.530386 (+ 214us) client_negotiation.cc:240] Negotiation successful
0812 00:40:10.530779 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":589,"threads_started":1}
I20260812 00:40:10.534425 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.512471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35500 (local address 127.7.244.126:41181)
0812 00:40:10.514262 (+ 1791us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.514268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.514285 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:40:10.514341 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.514346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.514464 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.514568 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.516305 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.517080 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.518907 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.519342 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.519479 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.523892 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.523900 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.523905 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.523945 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.527754 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.528653 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.528658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.528660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.528792 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.529213 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.529218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.529221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.529729 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.529887 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.533780 (+ 3893us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.534072 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1636,"thread_start_us":1504,"threads_started":1}
I20260812 00:40:10.535526 21708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597: Bootstrap starting.
I20260812 00:40:10.535844 22003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:10.539999 21708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:10.541364 21708 log.cc:824] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495210, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:40:10.545645 21708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597: No bootstrap required, opened a new log
I20260812 00:40:10.550377 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.531667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.7.244.126:41181)
0812 00:40:10.532281 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.532288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.532307 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:40:10.532352 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.532356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.532418 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.532516 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.534066 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.535389 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.537590 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.538305 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.538425 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.543706 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.543712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.543716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.543750 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.546895 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.547858 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.547868 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.547872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.548061 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.548530 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.548536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.548539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.548948 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.549243 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.549663 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.549926 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":320,"threads_started":1}
I20260812 00:40:10.552441 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.536711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:35939 (local address 127.0.0.1:32786)
0812 00:40:10.536934 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.536954 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:10.537074 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:10.537557 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:10.537562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:10.538069 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:10.538266 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.538279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.539709 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.539714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.541126 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.541147 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.542354 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.542360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.542498 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.543099 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:10.543123 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:10.545778 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:10.549634 (+ 3856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.549640 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.549644 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.550203 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.550707 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.550713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.550717 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.550872 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.551432 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:10.551438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:10.551607 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 00:40:10.551756 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 00:40:10.552080 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 00:40:10.553020 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.536784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.7.244.125:35939)
0812 00:40:10.536974 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.536980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.537003 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:40:10.537246 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.537253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.537383 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.537487 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.538459 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.539472 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.541346 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.541983 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.542122 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.546048 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.546055 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.546061 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.546109 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.549444 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.550372 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.550379 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.550383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.550546 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.550987 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.550993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.550996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.551280 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.551424 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.551883 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.552627 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 00:40:10.554088 22003 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:10.559003 21708 raft_consensus.cc:348] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } }
I20260812 00:40:10.559540 21708 raft_consensus.cc:729] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c928c479d5bc497a8c7a72e545255597, State: Initialized, Role: FOLLOWER
I20260812 00:40:10.560216 21708 consensus_queue.cc:260] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } }
I20260812 00:40:10.561453 22016 sys_catalog.cc:455] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.562031 22016 sys_catalog.cc:458] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495210, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
I20260812 00:40:10.563164 21708 sys_catalog.cc:564] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:10.569572 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.554850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51754)
0812 00:40:10.555196 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.555214 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:10.555315 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:10.555784 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:10.555790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:10.556283 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:10.556499 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.556511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.557853 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.557860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.558889 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.558906 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.560103 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.560109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.560369 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.561093 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:10.561122 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:10.563894 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:10.567007 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.567012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.567017 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.567407 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.567763 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.567768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.567772 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.567921 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.568465 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:10.568470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:10.568629 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 00:40:10.568792 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 00:40:10.569169 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
I20260812 00:40:10.569785 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.555263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51754 (local address 127.7.244.124:33461)
0812 00:40:10.555501 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.555507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.555526 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:40:10.555575 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.555580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.555647 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.555738 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.556650 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.557715 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.559065 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.559732 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.559882 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.564155 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.564162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.564167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.564207 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.566859 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.567528 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.567532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.567535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.567656 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.568024 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.568027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.568029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.568341 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.568484 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.568977 (+ 493us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.569393 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":113}
I20260812 00:40:10.571192 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.552355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.7.244.125:35939)
0812 00:40:10.553259 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.553265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.553282 (+ 17us) server_negotiation.cc:407] Connection header received
0812 00:40:10.553358 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.553363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.553428 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.553513 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.555014 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.556055 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.557506 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.558015 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.558145 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.563412 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.563419 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.563422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.563460 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.567466 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.568592 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.568598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.568601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.568787 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.569333 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.569339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.569343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.569745 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.569993 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.570653 (+ 660us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.570868 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":1594,"threads_started":1}
I20260812 00:40:10.576948 22003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577: Bootstrap starting.
I20260812 00:40:10.577389 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:40:10.580178 21667 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c928c479d5bc497a8c7a72e545255597"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35512
I20260812 00:40:10.580614 21667 raft_consensus.cc:482] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 00:40:10.580814 21667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:10.581671 22003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:10.583334 22003 log.cc:824] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577: Log is configured to *not* fsync() on all Append() calls
I20260812 00:40:10.584137 21667 raft_consensus.cc:504] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } }
I20260812 00:40:10.585816 21667 leader_election.cc:288] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [CANDIDATE]: Term 1 election: Requested vote from peers 5206af2599a542d59edfb89570bbc458 (127.7.244.125:35939), 75deb06bffdb4273863d49db34f1b577 (127.7.244.124:33461)
I20260812 00:40:10.588924 22003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577: No bootstrap required, opened a new log
W20260812 00:40:10.593930 22036 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:40:10.594101 22036 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 00:40:10.598601 21624 tablet.cc:2369] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:40:10.591483 21590 master_runner.cc:186] Error getting master registration for 127.7.244.125:35939: OK, instance_id { permanent_uuid: "5206af2599a542d59edfb89570bbc458" instance_seqno: 1786495209904326 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:40:10.603277 21590 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:40:10.603933 22003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } }
I20260812 00:40:10.604478 22003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75deb06bffdb4273863d49db34f1b577, State: Initialized, Role: FOLLOWER
I20260812 00:40:10.605100 22003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } }
I20260812 00:40:10.608755 22003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:10.608707 21950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c928c479d5bc497a8c7a72e545255597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75deb06bffdb4273863d49db34f1b577"
I20260812 00:40:10.616658 21950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:10.620486 21950 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c928c479d5bc497a8c7a72e545255597 in term 1.
I20260812 00:40:10.622805 21618 leader_election.cc:302] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75deb06bffdb4273863d49db34f1b577, c928c479d5bc497a8c7a72e545255597; no voters:
I20260812 00:40:10.623420 22016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:40:10.624960 22016 raft_consensus.cc:686] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [term 1 LEADER]: Becoming Leader. State: Replica: c928c479d5bc497a8c7a72e545255597, State: Running, Role: LEADER
I20260812 00:40:10.625705 22016 consensus_queue.cc:237] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } }
I20260812 00:40:10.626560 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.606841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.7.244.126:41181)
0812 00:40:10.607094 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.607101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.607697 (+ 596us) server_negotiation.cc:407] Connection header received
0812 00:40:10.608025 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.608032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.608144 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.608237 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.610744 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.611881 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.613995 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.614715 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.614832 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.618708 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.618716 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.618721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.618771 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.622584 (+ 3813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.623600 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.623606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.623610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.623786 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.624351 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.624390 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.624394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.624898 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.625216 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.625901 (+ 685us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.626165 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 00:40:10.632416 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.633800 22062 sys_catalog.cc:455] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c928c479d5bc497a8c7a72e545255597. Latest consensus state: current_term: 1 leader_uuid: "c928c479d5bc497a8c7a72e545255597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.634200 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:10.634289 22062 sys_catalog.cc:458] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:10.635694 22064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 00:40:10.639101 21859 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } attempt: 5
I20260812 00:40:10.642342 22064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:40:10.646468 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.608506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51770 (local address 127.7.244.124:33461)
0812 00:40:10.608677 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.608683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.627443 (+ 18760us) server_negotiation.cc:407] Connection header received
0812 00:40:10.627740 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.627746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.627831 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.627907 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.629461 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.630566 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.632459 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.633276 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.633429 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.638829 (+ 5400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.638837 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.638842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.638889 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.642674 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.643644 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.643650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.643654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.643835 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.644341 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.644346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.644350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.644787 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.645126 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.645806 (+ 680us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.646064 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37,"spinlock_wait_cycles":4352}
Aug 12 00:40:10 dist-test-slave-gjw7 krb5kdc[21559](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
I20260812 00:40:10.655071 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.607504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.7.244.125:35939)
0812 00:40:10.607680 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.607687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.610692 (+ 3005us) server_negotiation.cc:407] Connection header received
0812 00:40:10.610962 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.610968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.611060 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.611136 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.614557 (+ 3421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.615765 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.619756 (+ 3991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.620463 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.620574 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.627934 (+ 7360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.627941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.627945 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.627989 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.631607 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.637451 (+ 5844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.637458 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.637463 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.637691 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.638583 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.638588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.638592 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.639010 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.646854 (+ 7844us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.652385 (+ 5531us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.653468 (+ 1083us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 00:40:10.671010 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.639669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:33461 (local address 127.0.0.1:51772)
0812 00:40:10.640027 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:10.640046 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:10.640163 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:10.641157 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:10.641162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:10.641696 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:10.641931 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.641945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.643353 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.643359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.644687 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:10.644704 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.645890 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.645897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:10.646146 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.647166 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:10.647191 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:10.650186 (+ 2995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:10.666566 (+ 16380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.666575 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.666580 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.667301 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.668208 (+ 907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:10.668215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:10.668219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:10.668432 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:10.669741 (+ 1309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:10.669748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:10.669938 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 00:40:10.670104 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 00:40:10.670529 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260812 00:40:10.677867 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:10.640509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.7.244.124:33461)
0812 00:40:10.640702 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.640708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.640727 (+ 19us) server_negotiation.cc:407] Connection header received
0812 00:40:10.640781 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.640785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.640865 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.640958 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.642107 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.643176 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.644872 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:10.645540 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:10.645664 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:10.650450 (+ 4786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:10.650458 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:10.650463 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:10.650531 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:10.666233 (+ 15702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.667649 (+ 1416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.667658 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.667662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.667992 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:10.668644 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:10.668675 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:10.668679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:10.669519 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:10.676131 (+ 6612us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:10.676495 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 00:40:10.677022 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"spinlock_wait_cycles":6912}
I20260812 00:40:10.684401 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458: Bootstrap starting.
I20260812 00:40:10.689075 21859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:10.690672 21859 log.cc:824] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458: Log is configured to *not* fsync() on all Append() calls
I20260812 00:40:10.694523 21859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458: No bootstrap required, opened a new log
I20260812 00:40:10.698544 21950 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [term 1 FOLLOWER]: Refusing update from remote peer c928c479d5bc497a8c7a72e545255597: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:40:10.699622 22062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:40:10.708815 21859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } }
I20260812 00:40:10.712359 21859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5206af2599a542d59edfb89570bbc458, State: Initialized, Role: FOLLOWER
I20260812 00:40:10.713160 21859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } }
I20260812 00:40:10.717860 22079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.720580 22079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:10.720427 21859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:10.721727 21815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c928c479d5bc497a8c7a72e545255597" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5206af2599a542d59edfb89570bbc458"
I20260812 00:40:10.722288 21815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [term 0 FOLLOWER]: Advancing to term 1
W20260812 00:40:10.726269 21914 tablet.cc:2369] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:40:10.727577 21815 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c928c479d5bc497a8c7a72e545255597 in term 1.
I20260812 00:40:10.728374 21814 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [term 1 FOLLOWER]: Refusing update from remote peer c928c479d5bc497a8c7a72e545255597: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:40:10.731168 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c928c479d5bc497a8c7a72e545255597. Latest consensus state: current_term: 1 leader_uuid: "c928c479d5bc497a8c7a72e545255597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.731421 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:10.731635 22062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:40:10.736084 22067 mvcc.cc:205] Tried to move back new op lower bound from 7317484382985940992 to 7317484382731190272. Current Snapshot: MvccSnapshot[applied={T|T < 7317484382985940992}]
I20260812 00:40:10.746938 22016 sys_catalog.cc:455] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c928c479d5bc497a8c7a72e545255597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.747340 22016 sys_catalog.cc:458] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:10.750383 22064 catalog_manager.cc:1347] Generated new cluster ID: 343536237fad42d7aa37df989a0b4386
I20260812 00:40:10.750512 22064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:40:10.769043 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c928c479d5bc497a8c7a72e545255597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.769399 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:10.782845 22079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c928c479d5bc497a8c7a72e545255597. Latest consensus state: current_term: 1 leader_uuid: "c928c479d5bc497a8c7a72e545255597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.783238 22079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:10.785174 22064 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:40:10.789031 22016 sys_catalog.cc:455] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c928c479d5bc497a8c7a72e545255597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.789467 22016 sys_catalog.cc:458] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597 [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:10.790139 22064 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:40:10.806829 22089 mvcc.cc:205] Tried to move back new op lower bound from 7317484382985940992 to 7317484382731190272. Current Snapshot: MvccSnapshot[applied={T|T < 7317484382985940992}]
I20260812 00:40:10.806761 22079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c928c479d5bc497a8c7a72e545255597" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c928c479d5bc497a8c7a72e545255597" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 41181 } } peers { permanent_uuid: "5206af2599a542d59edfb89570bbc458" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 35939 } } peers { permanent_uuid: "75deb06bffdb4273863d49db34f1b577" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 33461 } } }
I20260812 00:40:10.807243 22079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:10.792344 21872 master_runner.cc:186] Error getting master registration for 127.7.244.125:35939: OK, instance_id { permanent_uuid: "5206af2599a542d59edfb89570bbc458" instance_seqno: 1786495209904326 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:40:10.809662 21872 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:40:10.828008 22100 catalog_manager.cc:1259] Loaded cluster ID: 343536237fad42d7aa37df989a0b4386
I20260812 00:40:10.837410 22100 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577: loading cluster ID for follower catalog manager: success
I20260812 00:40:10.838326 22064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c928c479d5bc497a8c7a72e545255597: Generated new TSK 0
I20260812 00:40:10.839473 22064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:40:10.840703 22116 catalog_manager.cc:1259] Loaded cluster ID: 343536237fad42d7aa37df989a0b4386
I20260812 00:40:10.840863 22116 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458: loading cluster ID for follower catalog manager: success
I20260812 00:40:10.841723 22100 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577: acquiring CA information for follower catalog manager: success
W20260812 00:40:10.843047 22100 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260812 00:40:10.845110 22116 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458: acquiring CA information for follower catalog manager: success
I20260812 00:40:10.847062 22116 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5206af2599a542d59edfb89570bbc458: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 00:40:10.996806 22012 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:10.987047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.7.244.126:41181)
0812 00:40:10.987313 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.987320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.993638 (+ 6318us) server_negotiation.cc:407] Connection header received
0812 00:40:10.994958 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.994969 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.995084 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.995171 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.996496 (+ 1325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35532: BlockingRecv error: recv got EOF from 127.0.0.1:35532 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260812 00:40:10.999264 21866 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:10.983336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.7.244.125:35939)
0812 00:40:10.983560 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.983566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:10.993097 (+ 9531us) server_negotiation.cc:407] Connection header received
0812 00:40:10.997467 (+ 4370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:10.997474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:10.997576 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:10.997655 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:10.998994 (+ 1339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32810: BlockingRecv error: recv got EOF from 127.0.0.1:32810 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 00:40:11.003275 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:10.994276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51780 (local address 127.7.244.124:33461)
0812 00:40:10.994477 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:10.994484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.000162 (+ 5678us) server_negotiation.cc:407] Connection header received
0812 00:40:11.000820 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.000827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.000939 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.001019 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.002965 (+ 1946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51780: BlockingRecv error: recv got EOF from 127.0.0.1:51780 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:40:11.057772 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.040941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32820 (local address 127.7.244.125:35939)
0812 00:40:11.041171 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.041178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.049267 (+ 8089us) server_negotiation.cc:407] Connection header received
0812 00:40:11.049914 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.049921 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.050175 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.050335 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.051112 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.052314 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.054444 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.055142 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.055247 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.056120 (+ 873us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.056890 (+ 770us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.057020 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.057279 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.057418 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 00:40:11.111671 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.093320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51788 (local address 127.7.244.124:33461)
0812 00:40:11.093550 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.093557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.109408 (+ 15851us) server_negotiation.cc:407] Connection header received
0812 00:40:11.109886 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.109893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.110041 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.110137 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.111400 (+ 1263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51788: BlockingRecv error: recv got EOF from 127.0.0.1:51788 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 00:40:11.160753 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.144558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32830 (local address 127.7.244.125:35939)
0812 00:40:11.144783 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.144790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.152320 (+ 7530us) server_negotiation.cc:407] Connection header received
0812 00:40:11.152910 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.152916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.153021 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.153090 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.153880 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.156014 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.158408 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.159096 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.159195 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.159867 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.160049 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.160114 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.160330 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.160429 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 00:40:11.207975 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.198487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.7.244.124:33461)
0812 00:40:11.198695 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.198701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.205759 (+ 7058us) server_negotiation.cc:407] Connection header received
0812 00:40:11.206344 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.206351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.206455 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.206539 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.207715 (+ 1176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51804: BlockingRecv error: recv got EOF from 127.0.0.1:51804 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 00:40:11.227985 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.219899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32832 (local address 127.7.244.125:35939)
0812 00:40:11.220090 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.220096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.220731 (+ 635us) server_negotiation.cc:407] Connection header received
0812 00:40:11.221096 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.221103 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.221230 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.221323 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.222124 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.223321 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.225529 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.226255 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.226367 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.227058 (+ 691us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.227229 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.227304 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.227552 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.227646 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 00:40:11.245537 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.242619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51810 (local address 127.7.244.124:33461)
0812 00:40:11.242867 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.242873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.243471 (+ 598us) server_negotiation.cc:407] Connection header received
0812 00:40:11.243822 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.243829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.243948 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.244033 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.245253 (+ 1220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51810: BlockingRecv error: recv got EOF from 127.0.0.1:51810 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 00:40:11.262844 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.254744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32842 (local address 127.7.244.125:35939)
0812 00:40:11.254956 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.254967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.255696 (+ 729us) server_negotiation.cc:407] Connection header received
0812 00:40:11.255954 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.255961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.256127 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.256267 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.256809 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.258112 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.260234 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.261064 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.261242 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.261846 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.262023 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.262136 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.262345 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.262451 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 00:40:11.274850 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.271853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51822 (local address 127.7.244.124:33461)
0812 00:40:11.272149 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.272155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.272748 (+ 593us) server_negotiation.cc:407] Connection header received
0812 00:40:11.273086 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.273092 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.273215 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.273331 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.274540 (+ 1209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51822: BlockingRecv error: recv got EOF from 127.0.0.1:51822 (error 108)
Metrics: {"server-negotiator.queue_time_us":145}
I20260812 00:40:11.299149 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.291436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.7.244.125:35939)
0812 00:40:11.291641 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.291647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.292284 (+ 637us) server_negotiation.cc:407] Connection header received
0812 00:40:11.292614 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.292621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.292723 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.292864 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.293401 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.294631 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.296677 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.297471 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.297618 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.298243 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.298413 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.298512 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.298705 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.298817 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260812 00:40:11.312651 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.309227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.7.244.124:33461)
0812 00:40:11.309456 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.309466 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.310866 (+ 1400us) server_negotiation.cc:407] Connection header received
0812 00:40:11.311155 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.311161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.311273 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.311412 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.312406 (+ 994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51826: BlockingRecv error: recv got EOF from 127.0.0.1:51826 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 00:40:11.343053 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.335513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.7.244.125:35939)
0812 00:40:11.335778 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.335785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.336433 (+ 648us) server_negotiation.cc:407] Connection header received
0812 00:40:11.336680 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.336686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.336785 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.336900 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.337451 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.338691 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.340714 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.341490 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.341640 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.342238 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.342399 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.342484 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.342654 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.342755 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260812 00:40:11.353503 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.350521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.7.244.124:33461)
0812 00:40:11.350929 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.350935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.351548 (+ 613us) server_negotiation.cc:407] Connection header received
0812 00:40:11.351890 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.351897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.352016 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.352091 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.353177 (+ 1086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51840: BlockingRecv error: recv got EOF from 127.0.0.1:51840 (error 108)
Metrics: {"server-negotiator.queue_time_us":247}
I20260812 00:40:11.398154 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.389889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.7.244.125:35939)
0812 00:40:11.390121 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.390128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.392883 (+ 2755us) server_negotiation.cc:407] Connection header received
0812 00:40:11.393216 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.393222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.393346 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.393455 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.394050 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.394961 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.396329 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.396825 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.396944 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.397414 (+ 470us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.397526 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.397602 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.397753 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.397849 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 00:40:11.407940 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.406221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51846 (local address 127.7.244.124:33461)
0812 00:40:11.406385 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.406389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.406761 (+ 372us) server_negotiation.cc:407] Connection header received
0812 00:40:11.406929 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.406934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.407007 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.407094 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.407742 (+ 648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51846: BlockingRecv error: recv got EOF from 127.0.0.1:51846 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 00:40:11.481640 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.475982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32880 (local address 127.7.244.125:35939)
0812 00:40:11.476192 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.476196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.476617 (+ 421us) server_negotiation.cc:407] Connection header received
0812 00:40:11.476771 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.476775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.476857 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.476943 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.477465 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.478300 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.479759 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.480185 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.480305 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.480868 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.480997 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.481077 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.481239 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.481364 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260812 00:40:11.490142 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.488006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51848 (local address 127.7.244.124:33461)
0812 00:40:11.488245 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.488253 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.488583 (+ 330us) server_negotiation.cc:407] Connection header received
0812 00:40:11.488786 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.488790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.488887 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.488977 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.489888 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51848: BlockingRecv error: recv got EOF from 127.0.0.1:51848 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 00:40:11.629984 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.624292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32892 (local address 127.7.244.125:35939)
0812 00:40:11.624500 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.624507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.624940 (+ 433us) server_negotiation.cc:407] Connection header received
0812 00:40:11.625158 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.625163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.625258 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.625357 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.625863 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.626683 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.628155 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.628593 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.628715 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.629190 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.629338 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.629414 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.629583 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.629675 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 00:40:11.638054 22007 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:11.636035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.7.244.124:33461)
0812 00:40:11.636288 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.636295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.636538 (+ 243us) server_negotiation.cc:407] Connection header received
0812 00:40:11.636754 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.636758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.636859 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.636954 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:11.637860 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51862: BlockingRecv error: recv got EOF from 127.0.0.1:51862 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 00:40:11.846087 22100 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 75deb06bffdb4273863d49db34f1b577: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:40:11.903944 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.898176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32902 (local address 127.7.244.125:35939)
0812 00:40:11.898428 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.898436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.898610 (+ 174us) server_negotiation.cc:407] Connection header received
0812 00:40:11.898869 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.898874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.898949 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.899031 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.899525 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.900273 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.901775 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.902267 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.902391 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.903081 (+ 690us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.903252 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.903342 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.903539 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.903659 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260812 00:40:11.916654 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.910564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51866 (local address 127.7.244.124:33461)
0812 00:40:11.910765 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.910772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.911183 (+ 411us) server_negotiation.cc:407] Connection header received
0812 00:40:11.911387 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.911393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.911573 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.911679 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.912122 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.912857 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.914300 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.914694 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.914816 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.915405 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.916007 (+ 602us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.916154 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.916273 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.916369 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 00:40:11.930928 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.923712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.7.244.125:35939)
0812 00:40:11.923971 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.923978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.924384 (+ 406us) server_negotiation.cc:407] Connection header received
0812 00:40:11.924607 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.924612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.924702 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.924784 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.925579 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.926592 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.928747 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.929433 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.929553 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.930051 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.930183 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.930271 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.930490 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.930588 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:40:11.931875 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.923683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.7.244.126:41181)
0812 00:40:11.924318 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.924324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.925239 (+ 915us) server_negotiation.cc:407] Connection header received
0812 00:40:11.925635 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.925641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.925809 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.925974 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.926779 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.927511 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.929358 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.930051 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.930175 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.930711 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.931160 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.931249 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.931487 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.931579 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":400,"threads_started":1}
I20260812 00:40:11.932090 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:11.925041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51872 (local address 127.7.244.124:33461)
0812 00:40:11.925261 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:11.925267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:11.925898 (+ 631us) server_negotiation.cc:407] Connection header received
0812 00:40:11.926066 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:11.926071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:11.926161 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:11.926253 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:11.926975 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.928071 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.930072 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:11.930640 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:11.930735 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:11.931400 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:11.931530 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:11.931614 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:11.931767 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 00:40:11.931854 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 00:40:11.938632 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21590
I20260812 00:40:11.939812 21701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:40:12.052618 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21590
I20260812 00:40:12.079978 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21718
I20260812 00:40:12.081408 21852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:40:12.189252 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21718
I20260812 00:40:12.219587 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 21872
I20260812 00:40:12.220738 21993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:40:12.332070 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 21872
2026-08-12T00:40:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5154 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 00:40:12 dist-test-slave-gjw7 krb5kdc[22353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 00:40:12 dist-test-slave-gjw7 krb5kdc[22353](info): set up 2 sockets
Aug 12 00:40:12 dist-test-slave-gjw7 krb5kdc[22353](info): commencing operation
krb5kdc: starting...
W20260812 00:40:13.688966 8145 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/test-user.keytab.
Aug 12 00:40:13 dist-test-slave-gjw7 krb5kdc[22353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T00:40:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T00:40:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.244.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
Entry for principal kudu/127.7.244.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab.
I20260812 00:40:14.036953 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:32997
--webserver_interface=127.7.244.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:32997,127.7.244.125:37433,127.7.244.124:42537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 00:40:14.404431 22384 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:40:14.404798 22384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:14.404896 22384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:14.405018 22384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:14.412647 22384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:14.412763 22384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:14.412824 22384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:14.412896 22384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:14.412995 22384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:14.423498 22384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:32997,127.7.244.125:37433,127.7.244.124:42537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.126.keytab
--principal=kudu/127.7.244.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.126:32997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.244.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:14.425055 22384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:14.426553 22384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:14.438943 22396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:14.438882 22397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:14.438971 22404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:14.439646 22384 server_base.cc:1034] running on GCE node
I20260812 00:40:14.440543 22384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:14.442149 22384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:14.443403 22384 hybrid_clock.cc:648] HybridClock initialized: now 1786495214443323 us; error 55 us; skew 500 ppm
I20260812 00:40:14.443763 22384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:14 dist-test-slave-gjw7 krb5kdc[22353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:14.452418 22384 init.cc:377] Logged in from keytab as kudu/127.7.244.126@KRBTEST.COM (short username kudu)
I20260812 00:40:14.454187 22384 webserver.cc:460] Webserver started at http://127.7.244.126:35523/ using document root <none> and password file <none>
I20260812 00:40:14.455009 22384 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:14.455121 22384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:14.455413 22384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:14.458712 22384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/data/instance:
uuid: "3ddbb20895654a9688d6038b9d0e65ce"
format_stamp: "Formatted at 2026-08-12 00:40:14 on dist-test-slave-gjw7"
server_key: "e482615336b999c558ccc8d4080b6b3a"
server_key_iv: "4787cb627fc7b0e4c7cfd91069312647"
server_key_version: "encryptionkey@0"
I20260812 00:40:14.459545 22384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/wal/instance:
uuid: "3ddbb20895654a9688d6038b9d0e65ce"
format_stamp: "Formatted at 2026-08-12 00:40:14 on dist-test-slave-gjw7"
server_key: "e482615336b999c558ccc8d4080b6b3a"
server_key_iv: "4787cb627fc7b0e4c7cfd91069312647"
server_key_version: "encryptionkey@0"
I20260812 00:40:14.465480 22384 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260812 00:40:14.469436 22417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:14.471063 22384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 00:40:14.471256 22384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/wal
uuid: "3ddbb20895654a9688d6038b9d0e65ce"
format_stamp: "Formatted at 2026-08-12 00:40:14 on dist-test-slave-gjw7"
server_key: "e482615336b999c558ccc8d4080b6b3a"
server_key_iv: "4787cb627fc7b0e4c7cfd91069312647"
server_key_version: "encryptionkey@0"
I20260812 00:40:14.471487 22384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:14.492489 22384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:14.497030 22384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:14.497334 22384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:14.516938 22384 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.126:32997
I20260812 00:40:14.516960 22520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.126:32997 every 8 connection(s)
I20260812 00:40:14.518508 22384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/data/info.pb
I20260812 00:40:14.521507 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 22384
I20260812 00:40:14.521714 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-0/wal/instance
I20260812 00:40:14.522193 8145 external_mini_cluster.cc:1428] Setting key cea84b791c93b3ef72e6e2fe22214110
I20260812 00:40:14.523051 22523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:14.530263 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:40:14 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.244.125@KRBTEST.COM; defaulting to no policy
I20260812 00:40:14.556696 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.533685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:32997 (local address 127.0.0.1:36670)
0812 00:40:14.534665 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.534735 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:14.537107 (+ 2372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:14.538228 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:14.538233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:14.538904 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:14.539282 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:14.539303 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:14.541611 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:14.541616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:14.542789 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:14.542805 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:14.544164 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:14.544170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:14.544320 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:14.545067 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:14.545092 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:14.547949 (+ 2857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:14.551837 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:14.551855 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:14.551872 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:14.552539 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:14.553037 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:14.553042 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:14.553046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:14.553336 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:14.554247 (+ 911us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:14.554257 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:14.554542 (+ 285us) client_negotiation.cc:769] Sending connection context
0812 00:40:14.554911 (+ 369us) client_negotiation.cc:240] Negotiation successful
0812 00:40:14.555375 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":387,"threads_started":1}
I20260812 00:40:14.556711 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.533686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.7.244.126:32997)
0812 00:40:14.535576 (+ 1890us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:14.535582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:14.535626 (+ 44us) server_negotiation.cc:407] Connection header received
0812 00:40:14.537590 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:14.537599 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:14.537973 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:14.538192 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:14.539538 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:14.541390 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:14.542974 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:14.543860 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:14.544128 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:14.548299 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:14.548311 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:14.548327 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:14.548366 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:14.551625 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:14.552707 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:14.552715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:14.552718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:14.552897 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:14.553464 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:14.553468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:14.553470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:14.554075 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:14.554258 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:14.555107 (+ 849us) server_negotiation.cc:299] Negotiation successful
0812 00:40:14.555559 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":157,"server-negotiator.queue_time_us":1682,"spinlock_wait_cycles":18176,"thread_start_us":1421,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.125@KRBTEST.COM" created.
I20260812 00:40:14.559981 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:14.561179 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.560700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54176)
0812 00:40:14.560891 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.561030 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 00:40:14.561491 22418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
I20260812 00:40:14.563223 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.562792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54192)
0812 00:40:14.562966 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.563055 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 00:40:14.563485 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:37433: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab.
Entry for principal kudu/127.7.244.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab.
I20260812 00:40:14.596825 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.125:37433
--webserver_interface=127.7.244.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab
--principal=kudu/127.7.244.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:32997,127.7.244.125:37433,127.7.244.124:42537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:40:14.612912 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } attempt: 1
I20260812 00:40:14.614362 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.613827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54208)
0812 00:40:14.614062 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.614171 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 00:40:14.615497 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.615095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54216)
0812 00:40:14.615245 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.615323 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 00:40:14.615705 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:37433: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
I20260812 00:40:14.683955 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } attempt: 2
I20260812 00:40:14.685281 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.684762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54224)
0812 00:40:14.684994 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.685089 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260812 00:40:14.686422 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.686019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54228)
0812 00:40:14.686176 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.686276 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 00:40:14.686625 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:37433: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
I20260812 00:40:14.777887 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } attempt: 3
I20260812 00:40:14.779134 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.778645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54242)
0812 00:40:14.778884 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.778986 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 00:40:14.780189 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.779805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54258)
0812 00:40:14.779955 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.780030 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 00:40:14.780395 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:37433: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
I20260812 00:40:14.923662 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } attempt: 4
I20260812 00:40:14.925010 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.924434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54260)
0812 00:40:14.924770 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.924867 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260812 00:40:14.926016 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:14.925653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54268)
0812 00:40:14.925800 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:14.925883 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 00:40:14.926231 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.125:37433: Network error: Client connection negotiation failed: client connection to 127.7.244.125:37433: connect: Connection refused (error 111)
W20260812 00:40:14.962167 22538 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:40:14.962586 22538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:14.962689 22538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:14.962806 22538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:14.970633 22538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:14.970759 22538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:14.970842 22538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:14.970912 22538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:14.971002 22538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:14.981760 22538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:32997,127.7.244.125:37433,127.7.244.124:42537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.125.keytab
--principal=kudu/127.7.244.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.125:37433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.244.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:14.983225 22538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:14.984685 22538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:14.997071 22552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:14.997586 22557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:14.997099 22554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:14.998801 22538 server_base.cc:1034] running on GCE node
I20260812 00:40:14.999745 22538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:15.001466 22538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:15.002799 22538 hybrid_clock.cc:648] HybridClock initialized: now 1786495215002670 us; error 65 us; skew 500 ppm
I20260812 00:40:15.003217 22538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:15.006898 22538 init.cc:377] Logged in from keytab as kudu/127.7.244.125@KRBTEST.COM (short username kudu)
I20260812 00:40:15.008615 22538 webserver.cc:460] Webserver started at http://127.7.244.125:40161/ using document root <none> and password file <none>
I20260812 00:40:15.009445 22538 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:15.009570 22538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:15.009927 22538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:15.013073 22538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/data/instance:
uuid: "bd817d834c084a3ba25657bf987529e2"
format_stamp: "Formatted at 2026-08-12 00:40:15 on dist-test-slave-gjw7"
server_key: "810f69c571b6b82e52736556440cad7d"
server_key_iv: "54eb82293fb370f77efc71894643a61a"
server_key_version: "encryptionkey@0"
I20260812 00:40:15.014037 22538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/wal/instance:
uuid: "bd817d834c084a3ba25657bf987529e2"
format_stamp: "Formatted at 2026-08-12 00:40:15 on dist-test-slave-gjw7"
server_key: "810f69c571b6b82e52736556440cad7d"
server_key_iv: "54eb82293fb370f77efc71894643a61a"
server_key_version: "encryptionkey@0"
I20260812 00:40:15.019604 22538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 00:40:15.023674 22575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:15.025319 22538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 00:40:15.025542 22538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/wal
uuid: "bd817d834c084a3ba25657bf987529e2"
format_stamp: "Formatted at 2026-08-12 00:40:15 on dist-test-slave-gjw7"
server_key: "810f69c571b6b82e52736556440cad7d"
server_key_iv: "54eb82293fb370f77efc71894643a61a"
server_key_version: "encryptionkey@0"
I20260812 00:40:15.025779 22538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:15.043017 22538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:15.047808 22538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:15.048080 22538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:15.066744 22538 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.125:37433
I20260812 00:40:15.066762 22676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.125:37433 every 8 connection(s)
I20260812 00:40:15.068562 22538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/data/info.pb
I20260812 00:40:15.071542 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 22538
I20260812 00:40:15.071710 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-1/wal/instance
I20260812 00:40:15.072156 8145 external_mini_cluster.cc:1428] Setting key ab2543ef5b9c920478594f7c6e268757
I20260812 00:40:15.072949 22678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:15.080385 22678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:40:15.104965 22681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.083337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36686 (local address 127.7.244.126:32997)
0812 00:40:15.083931 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.083937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.084945 (+ 1008us) server_negotiation.cc:407] Connection header received
0812 00:40:15.087467 (+ 2522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.087472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.087551 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.087654 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.089961 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.090739 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.093079 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.093566 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.093681 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.098398 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.098405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.098408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.098453 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.101344 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.102450 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.102453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.102456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.102587 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.103080 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.103083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.103085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.103378 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.103492 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.104489 (+ 997us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.104681 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":290,"threads_started":1}
WARNING: no policy specified for kudu/127.7.244.124@KRBTEST.COM; defaulting to no policy
I20260812 00:40:15.106487 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.083874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:32997 (local address 127.0.0.1:36686)
0812 00:40:15.084747 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.084806 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:15.087099 (+ 2293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:15.087957 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:15.087968 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:15.088845 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:15.089702 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.089732 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.090936 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.090943 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.092915 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.092937 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.093831 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.093839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.094139 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.094960 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:15.094997 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:15.098062 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:15.101532 (+ 3470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.101544 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.101566 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.102324 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.102692 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.102698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.102701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.102975 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.103512 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:15.103522 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:15.104035 (+ 513us) client_negotiation.cc:769] Sending connection context
0812 00:40:15.104389 (+ 354us) client_negotiation.cc:240] Negotiation successful
0812 00:40:15.104976 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":355,"threads_started":1}
I20260812 00:40:15.108369 22678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.244.124@KRBTEST.COM" created.
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:40:15.123581 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.109053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54276)
0812 00:40:15.109368 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.109388 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:15.109508 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:15.110586 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:15.110590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:15.111062 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:15.111201 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.111211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.112609 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.112615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.113546 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.113560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.114414 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.114420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.114588 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.115272 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:15.115295 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:15.117725 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:15.120978 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.120984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.120988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.121413 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.121795 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.121800 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.121804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.121964 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.122628 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:15.122633 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:15.122774 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 00:40:15.122895 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 00:40:15.123193 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260812 00:40:15.123713 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.109428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54276 (local address 127.7.244.125:37433)
0812 00:40:15.110136 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.110142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.110170 (+ 28us) server_negotiation.cc:407] Connection header received
0812 00:40:15.110219 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.110226 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.110424 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.110574 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.111379 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.112466 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.113686 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.114265 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.114379 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.117963 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.117979 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.117996 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.118031 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.120827 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.121550 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.121557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.121560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.121683 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.122073 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.122076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.122078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.122471 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.122686 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.123109 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.123345 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":399,"threads_started":1}
I20260812 00:40:15.126124 22678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:15.127377 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.126803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43450)
0812 00:40:15.127024 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.127176 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260812 00:40:15.127665 22585 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
I20260812 00:40:15.129470 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.128971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43460)
0812 00:40:15.129186 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.129328 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 00:40:15.129695 22678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.244.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab.
Entry for principal kudu/127.7.244.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab.
I20260812 00:40:15.148847 8145 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
/tmp/dist-test-taskOGbh_J/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.124:42537
--webserver_interface=127.7.244.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab
--principal=kudu/127.7.244.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.244.84:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.244.126:32997,127.7.244.125:37433,127.7.244.124:42537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 00:40:15.179066 22678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 1
I20260812 00:40:15.180428 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.179925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43464)
0812 00:40:15.180142 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.180258 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 00:40:15.181524 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.181134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43476)
0812 00:40:15.181282 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.181407 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 00:40:15.181749 22678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
I20260812 00:40:15.225518 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } attempt: 5
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:40:15.240476 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.226559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54282 (local address 127.7.244.125:37433)
0812 00:40:15.226777 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.226783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.226809 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:40:15.226882 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.226888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.226997 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.227105 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.227933 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.229064 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.230513 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.231233 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.231387 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.234896 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.234902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.234906 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.234946 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.237763 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.238514 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.238519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.238521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.238652 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.239041 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.239044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.239046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.239357 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.239493 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.239917 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.240137 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 00:40:15.240487 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.226357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54282)
0812 00:40:15.226588 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.226611 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:15.226745 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:15.227140 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:15.227144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:15.227543 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:15.227723 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.227733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.229213 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.229217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.230321 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.230340 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.231460 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.231466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.231666 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.232344 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:15.232364 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:15.234636 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:15.237945 (+ 3309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.237952 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.237957 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.238377 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.238780 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.238786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.238790 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.238934 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.239476 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:15.239481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:15.239643 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 00:40:15.239802 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 00:40:15.240120 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 00:40:15.241820 22523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:15.243141 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.242726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43480)
0812 00:40:15.242913 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.242994 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 00:40:15.244151 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.243817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43482)
0812 00:40:15.243961 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.244046 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 00:40:15.244403 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
I20260812 00:40:15.249985 22678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 2
I20260812 00:40:15.251157 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.250687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43496)
0812 00:40:15.250891 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.250989 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 00:40:15.252317 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.251948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43510)
0812 00:40:15.252099 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.252181 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 00:40:15.252553 22678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
I20260812 00:40:15.295717 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 1
I20260812 00:40:15.297029 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.296525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43520)
0812 00:40:15.296742 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.296835 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 00:40:15.298223 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.297878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43522)
0812 00:40:15.298021 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.298102 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 00:40:15.298465 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
I20260812 00:40:15.343801 22678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 3
I20260812 00:40:15.345067 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.344557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43534)
0812 00:40:15.344789 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.344904 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 00:40:15.346202 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.345775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43548)
0812 00:40:15.346008 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.346090 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260812 00:40:15.346429 22678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
I20260812 00:40:15.366709 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 2
I20260812 00:40:15.368117 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.367577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43552)
0812 00:40:15.367838 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.367948 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260812 00:40:15.369159 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.368805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43562)
0812 00:40:15.368955 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.369030 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 00:40:15.369429 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
I20260812 00:40:15.475653 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 3
I20260812 00:40:15.476994 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.476473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43578)
0812 00:40:15.476703 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.476800 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260812 00:40:15.478289 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.477732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43582)
0812 00:40:15.478051 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.478148 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
W20260812 00:40:15.478546 22523 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
I20260812 00:40:15.489650 22678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 4
I20260812 00:40:15.490837 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.490329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43594)
0812 00:40:15.490581 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.490673 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 00:40:15.491900 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.491558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43598)
0812 00:40:15.491709 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.491784 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 00:40:15.492103 22678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.244.124:42537: Network error: Client connection negotiation failed: client connection to 127.7.244.124:42537: connect: Connection refused (error 111)
W20260812 00:40:15.518169 22688 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 00:40:15.518625 22688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 00:40:15.518720 22688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 00:40:15.518846 22688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 00:40:15.530087 22688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 00:40:15.530220 22688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 00:40:15.530303 22688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 00:40:15.530375 22688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 00:40:15.530470 22688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 00:40:15.541026 22688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.244.84:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.244.126:32997,127.7.244.125:37433,127.7.244.124:42537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/krb5kdc/kudu_127.7.244.124.keytab
--principal=kudu/127.7.244.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.244.124:42537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.244.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 22:51:51 UTC on ccycloud.quasar-sbbrqv.root.comops.site
build id 4581606
ASAN enabled
I20260812 00:40:15.542474 22688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 00:40:15.543879 22688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 00:40:15.556658 22703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:15.556586 22706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 00:40:15.556623 22701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 00:40:15.557968 22688 server_base.cc:1034] running on GCE node
I20260812 00:40:15.558887 22688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 00:40:15.560474 22688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 00:40:15.561712 22688 hybrid_clock.cc:648] HybridClock initialized: now 1786495215561607 us; error 73 us; skew 500 ppm
I20260812 00:40:15.562085 22688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 00:40:15.565656 22688 init.cc:377] Logged in from keytab as kudu/127.7.244.124@KRBTEST.COM (short username kudu)
I20260812 00:40:15.567454 22688 webserver.cc:460] Webserver started at http://127.7.244.124:34303/ using document root <none> and password file <none>
I20260812 00:40:15.568253 22688 fs_manager.cc:359] Metadata directory not provided
I20260812 00:40:15.568374 22688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 00:40:15.568737 22688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 00:40:15.572230 22688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/data/instance:
uuid: "d369d78824794d30b3ce4d051bb14545"
format_stamp: "Formatted at 2026-08-12 00:40:15 on dist-test-slave-gjw7"
server_key: "7e1a2c91c10b8b060c5e567129c77618"
server_key_iv: "7e1bba71f3c25acd8db2a5bdcc57774e"
server_key_version: "encryptionkey@0"
I20260812 00:40:15.573201 22688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/wal/instance:
uuid: "d369d78824794d30b3ce4d051bb14545"
format_stamp: "Formatted at 2026-08-12 00:40:15 on dist-test-slave-gjw7"
server_key: "7e1a2c91c10b8b060c5e567129c77618"
server_key_iv: "7e1bba71f3c25acd8db2a5bdcc57774e"
server_key_version: "encryptionkey@0"
I20260812 00:40:15.579427 22688 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260812 00:40:15.583616 22721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 00:40:15.585363 22688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 00:40:15.585590 22688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/data,/tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/wal
uuid: "d369d78824794d30b3ce4d051bb14545"
format_stamp: "Formatted at 2026-08-12 00:40:15 on dist-test-slave-gjw7"
server_key: "7e1a2c91c10b8b060c5e567129c77618"
server_key_iv: "7e1bba71f3c25acd8db2a5bdcc57774e"
server_key_version: "encryptionkey@0"
I20260812 00:40:15.585835 22688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 00:40:15.616832 22688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 00:40:15.621618 22688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 00:40:15.621907 22688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 00:40:15.640952 22688 rpc_server.cc:307] RPC server started. Bound to: 127.7.244.124:42537
I20260812 00:40:15.640972 22815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.244.124:42537 every 8 connection(s)
I20260812 00:40:15.642449 22688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/data/info.pb
I20260812 00:40:15.643138 8145 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu as pid 22688
I20260812 00:40:15.643366 8145 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOGbh_J/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786494824837501-8145-0/minicluster-data/master-2/wal/instance
I20260812 00:40:15.643954 8145 external_mini_cluster.cc:1428] Setting key 543006bbeb21a12c26747c5b03ed5c32
I20260812 00:40:15.647151 22816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 00:40:15.654495 22816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:15.655750 22523 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 4
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.126@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:40:15.669962 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.645922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43610 (local address 127.7.244.124:42537)
0812 00:40:15.646765 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.646779 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.646822 (+ 43us) server_negotiation.cc:407] Connection header received
0812 00:40:15.648485 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.648510 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.648946 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.649373 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.650970 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.652367 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.653716 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.654391 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.654578 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.659869 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.659882 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.659908 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.659966 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.663902 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.664868 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.664878 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.664889 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.665128 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.665683 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.665689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.665693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.666510 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.666871 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.667384 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.668389 (+ 1005us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":360,"threads_started":1}
I20260812 00:40:15.673830 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.656450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43622)
0812 00:40:15.656752 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.656773 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:15.656898 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:15.658659 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:15.658664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:15.659178 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:15.659382 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.659395 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.660728 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.660733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.662264 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.662277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.663017 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.663022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.663233 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.663778 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:15.663797 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:15.666599 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:15.670829 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.670836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.670841 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.671275 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.671713 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.671720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.671724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.671907 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.672627 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:15.672633 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:15.672839 (+ 206us) client_negotiation.cc:769] Sending connection context
0812 00:40:15.673055 (+ 216us) client_negotiation.cc:240] Negotiation successful
0812 00:40:15.673420 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":1}
I20260812 00:40:15.674782 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.656487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.7.244.124:42537)
0812 00:40:15.658302 (+ 1815us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.658311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.658334 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:40:15.658387 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.658392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.658462 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.659169 (+ 707us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.659554 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.660556 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.662398 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.662864 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.662972 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.666853 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.666860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.666864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.666901 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.670660 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.671413 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.671419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.671421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.671546 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.672035 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.672040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.672043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.672453 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.672614 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.674136 (+ 1522us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.674381 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1677,"thread_start_us":636,"threads_started":1}
I20260812 00:40:15.675279 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.657498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.126:32997 (local address 127.0.0.1:36698)
0812 00:40:15.658355 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.658403 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:15.658583 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:15.659090 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:15.659098 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:15.660124 (+ 1026us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:15.660527 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.660540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.662173 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.662179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.663923 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.663935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.664804 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.664811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.664993 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.665645 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:15.665669 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:15.668302 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:15.672113 (+ 3811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.672122 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.672128 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.672567 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.673049 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.673054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.673057 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.673221 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.673848 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:15.673856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:15.674077 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 00:40:15.674334 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 00:40:15.674674 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":424,"threads_started":1}
I20260812 00:40:15.675637 22824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.657461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.7.244.126:32997)
0812 00:40:15.658002 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.658008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.658701 (+ 693us) server_negotiation.cc:407] Connection header received
0812 00:40:15.658764 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.658769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.658871 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.658991 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.660715 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.661961 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.664094 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.664611 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.664727 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.668524 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.668530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.668533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.668566 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.671969 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.672712 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.672717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.672721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.672905 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.673383 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.673389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.673392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.673709 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.673853 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.674984 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.675264 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":403,"spinlock_wait_cycles":74496,"thread_start_us":237,"threads_started":1}
I20260812 00:40:15.677175 22816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:15.680732 22523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce: Bootstrap starting.
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.126@KRBTEST.COM
I20260812 00:40:15.684907 22523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:15.686576 22523 log.cc:824] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce: Log is configured to *not* fsync() on all Append() calls
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:40:15.690943 22523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce: No bootstrap required, opened a new log
I20260812 00:40:15.693759 22833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.674080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36712 (local address 127.7.244.126:32997)
0812 00:40:15.674668 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.674673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.674814 (+ 141us) server_negotiation.cc:407] Connection header received
0812 00:40:15.675047 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.675053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.675131 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.677056 (+ 1925us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.677442 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.678552 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.679834 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.680312 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.680440 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.685591 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.685599 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.685604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.685659 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.689043 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.690019 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.690025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.690029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.690216 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.690736 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.690741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.690744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.691177 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.691463 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.693030 (+ 1567us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.693283 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":288,"threads_started":1}
I20260812 00:40:15.695711 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.677958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.125:37433 (local address 127.0.0.1:54298)
0812 00:40:15.678197 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.678216 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:15.678311 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:15.681613 (+ 3302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:15.681619 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:15.682120 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:15.682329 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.682341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.683748 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.683754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.684716 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.684732 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.685685 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.685692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.685921 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.686564 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:15.686589 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:15.689045 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:15.692613 (+ 3568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.692620 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.692624 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.693197 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.693774 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.693780 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.693784 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.693977 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.694703 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:15.694708 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:15.694889 (+ 181us) client_negotiation.cc:769] Sending connection context
0812 00:40:15.695042 (+ 153us) client_negotiation.cc:240] Negotiation successful
0812 00:40:15.695355 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260812 00:40:15.696226 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.678089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54298 (local address 127.7.244.125:37433)
0812 00:40:15.681076 (+ 2987us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.681083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.681106 (+ 23us) server_negotiation.cc:407] Connection header received
0812 00:40:15.681171 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.681175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.681255 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.681385 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.682539 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.683574 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.684879 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.685490 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.685632 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.689449 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.689458 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.689462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.689509 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.692450 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.693407 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.693413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.693417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.693600 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.694113 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.694118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.694122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.694465 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.694602 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.695168 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.695416 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2831}
I20260812 00:40:15.697587 22816 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } has no permanent_uuid. Determining permanent_uuid...
I20260812 00:40:15.701164 22523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } }
I20260812 00:40:15.701678 22523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ddbb20895654a9688d6038b9d0e65ce, State: Initialized, Role: FOLLOWER
I20260812 00:40:15.702265 22523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } }
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.244.125@KRBTEST.COM
I20260812 00:40:15.705808 22841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:15.706187 22841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.124@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
I20260812 00:40:15.709076 22523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:15.714848 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.698539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43632)
0812 00:40:15.698777 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.698794 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:15.698915 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:15.699254 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:15.699259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:15.699743 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:15.699973 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.699985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.701342 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.701348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.702908 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.702927 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.703926 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.703933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.704180 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.704851 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:15.704885 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:15.707561 (+ 2676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:15.711611 (+ 4050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.711618 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.711623 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.712223 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.712692 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.712696 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.712699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.712869 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.713610 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:15.713616 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:15.713802 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 00:40:15.714021 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 00:40:15.714421 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260812 00:40:15.715564 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.695885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.7.244.125:37433)
0812 00:40:15.696628 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.696633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.696659 (+ 26us) server_negotiation.cc:407] Connection header received
0812 00:40:15.696840 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.696846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.696914 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.697002 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.698659 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.699692 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.701385 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.701949 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.702070 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.707410 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.707417 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.707421 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.707463 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.711401 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.712530 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.712536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.712539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.712708 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.713254 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.713259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.713262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.713660 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.713915 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.714384 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.714627 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":400,"threads_started":1}
I20260812 00:40:15.715700 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.698463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43632 (local address 127.7.244.124:42537)
0812 00:40:15.698641 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.698647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.698865 (+ 218us) server_negotiation.cc:407] Connection header received
0812 00:40:15.699016 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.699020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.699109 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.699207 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.700130 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.701158 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.703087 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.703727 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.703891 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.707789 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.707795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.707800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.707838 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.711388 (+ 3550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.712384 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.712389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.712392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.712561 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.712983 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.712987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.712990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.713472 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.713628 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.714148 (+ 520us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.715376 (+ 1228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":942,"server-negotiator.queue_time_us":67}
W20260812 00:40:15.721820 22863 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 00:40:15.721992 22863 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:40:15.721833 22816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: Bootstrap starting.
I20260812 00:40:15.725490 8145 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 00:40:15.726228 22816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:15.727567 22816 log.cc:824] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: Log is configured to *not* fsync() on all Append() calls
I20260812 00:40:15.728543 22466 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3ddbb20895654a9688d6038b9d0e65ce"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36712
I20260812 00:40:15.728957 22466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 00:40:15.729164 22466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:15.732971 22466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } }
I20260812 00:40:15.735777 22466 leader_election.cc:288] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [CANDIDATE]: Term 1 election: Requested vote from peers bd817d834c084a3ba25657bf987529e2 (127.7.244.125:37433), d369d78824794d30b3ce4d051bb14545 (127.7.244.124:42537)
I20260812 00:40:15.737560 22816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: No bootstrap required, opened a new log
I20260812 00:40:15.722435 22384 master_runner.cc:186] Error getting master registration for 127.7.244.125:37433: OK, instance_id { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" instance_seqno: 1786495215061039 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:40:15.739753 22384 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:40:15.749723 22816 raft_consensus.cc:348] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } }
I20260812 00:40:15.750332 22816 raft_consensus.cc:729] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d369d78824794d30b3ce4d051bb14545, State: Initialized, Role: FOLLOWER
W20260812 00:40:15.750681 22429 tablet.cc:2369] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:40:15.750953 22816 consensus_queue.cc:260] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } }
I20260812 00:40:15.752537 22874 sys_catalog.cc:455] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:15.753580 22874 sys_catalog.cc:458] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:15.753737 22816 sys_catalog.cc:564] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:15.753791 22770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ddbb20895654a9688d6038b9d0e65ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d369d78824794d30b3ce4d051bb14545"
I20260812 00:40:15.754493 22770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:15.760236 22770 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ddbb20895654a9688d6038b9d0e65ce in term 1.
I20260812 00:40:15.761260 22423 leader_election.cc:302] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ddbb20895654a9688d6038b9d0e65ce, d369d78824794d30b3ce4d051bb14545; no voters:
I20260812 00:40:15.761991 22841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [term 1 FOLLOWER]: Leader election won for term 1
I20260812 00:40:15.773664 22841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [term 1 LEADER]: Becoming Leader. State: Replica: 3ddbb20895654a9688d6038b9d0e65ce, State: Running, Role: LEADER
I20260812 00:40:15.774534 22841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } }
I20260812 00:40:15.787151 22883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ddbb20895654a9688d6038b9d0e65ce. Latest consensus state: current_term: 1 leader_uuid: "3ddbb20895654a9688d6038b9d0e65ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:15.787853 22883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:15.791285 22678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } attempt: 5
I20260812 00:40:15.794821 22901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 00:40:15 dist-test-slave-gjw7 krb5kdc[22353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786495215, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.244.125@KRBTEST.COM for kudu/127.7.244.124@KRBTEST.COM
I20260812 00:40:15.802891 22901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 00:40:15.812297 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.792136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.7.244.124:42537)
0812 00:40:15.792340 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.792346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.792424 (+ 78us) server_negotiation.cc:407] Connection header received
0812 00:40:15.792652 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.792659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.792747 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.792845 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.794509 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.795710 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.798134 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.798836 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.798971 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.803712 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.803720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.803726 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.803776 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.807812 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.808977 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.808984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.808989 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.809182 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.809795 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.809801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.809805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.810227 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.810388 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.811423 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.811826 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 00:40:15.813207 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.792046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.244.124:42537 (local address 127.0.0.1:43640)
0812 00:40:15.792336 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 00:40:15.792356 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 00:40:15.792511 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 00:40:15.793465 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 00:40:15.793471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 00:40:15.794047 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0812 00:40:15.794298 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.794313 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.795917 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.795941 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.797903 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 00:40:15.797927 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.799198 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.799206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 00:40:15.799465 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.800190 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 00:40:15.800218 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 00:40:15.803455 (+ 3237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 00:40:15.808059 (+ 4604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.808065 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.808070 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.808783 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.809349 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 00:40:15.809355 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 00:40:15.809359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 00:40:15.809543 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 00:40:15.810462 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 00:40:15.810467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 00:40:15.810649 (+ 182us) client_negotiation.cc:769] Sending connection context
0812 00:40:15.810792 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 00:40:15.812758 (+ 1966us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260812 00:40:15.820654 22833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.770828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.7.244.126:32997)
0812 00:40:15.771063 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.771071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.779226 (+ 8155us) server_negotiation.cc:407] Connection header received
0812 00:40:15.780689 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.780696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.780816 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.780901 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.788973 (+ 8072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.790269 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.795413 (+ 5144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.796189 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.796313 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.810101 (+ 13788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.810110 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.810116 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.810170 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.815753 (+ 5583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.816952 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.816958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.816962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.817152 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.817737 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.817742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.817745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.818194 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.819702 (+ 1508us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.819897 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.820210 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 00:40:15.829366 22678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2: Bootstrap starting.
I20260812 00:40:15.834555 22770 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [term 1 FOLLOWER]: Refusing update from remote peer 3ddbb20895654a9688d6038b9d0e65ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:40:15.835836 22883 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:40:15.845155 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.777356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43638 (local address 127.7.244.124:42537)
0812 00:40:15.777563 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.777569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.793843 (+ 16274us) server_negotiation.cc:407] Connection header received
0812 00:40:15.797442 (+ 3599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.797449 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.797556 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.797636 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.803126 (+ 5490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.804228 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.811073 (+ 6845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.811944 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.812073 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.831426 (+ 19353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.831435 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.831440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.831496 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.837030 (+ 5534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.839415 (+ 2385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.839421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.839425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.839631 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.841344 (+ 1713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.841353 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.841356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.841759 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.844234 (+ 2475us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.844419 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.844714 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 00:40:15.849392 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:15.779664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54316 (local address 127.7.244.125:37433)
0812 00:40:15.779881 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:15.779887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:15.800695 (+ 20808us) server_negotiation.cc:407] Connection header received
0812 00:40:15.800952 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:15.800958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:15.801082 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:15.801163 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:15.806093 (+ 4930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.807091 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.823952 (+ 16861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:15.824794 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:15.824923 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:15.828857 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 00:40:15.828864 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 00:40:15.828869 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 00:40:15.828920 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 00:40:15.838258 (+ 9338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.840574 (+ 2316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.840580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.840584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.840769 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 00:40:15.842300 (+ 1531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 00:40:15.842306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 00:40:15.842310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 00:40:15.842711 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 00:40:15.848507 (+ 5796us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:15.848691 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 00:40:15.848989 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 00:40:15.854592 22678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2: Neither blocks nor log segments found. Creating new log.
I20260812 00:40:15.856258 22678 log.cc:824] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2: Log is configured to *not* fsync() on all Append() calls
I20260812 00:40:15.866961 22678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2: No bootstrap required, opened a new log
W20260812 00:40:15.874141 22730 tablet.cc:2369] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 00:40:15.882218 22678 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } }
I20260812 00:40:15.883715 22874 sys_catalog.cc:455] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ddbb20895654a9688d6038b9d0e65ce. Latest consensus state: current_term: 1 leader_uuid: "3ddbb20895654a9688d6038b9d0e65ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:15.884074 22874 sys_catalog.cc:458] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:15.890475 22678 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd817d834c084a3ba25657bf987529e2, State: Initialized, Role: FOLLOWER
I20260812 00:40:15.891294 22678 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } }
I20260812 00:40:15.892637 22904 mvcc.cc:205] Tried to move back new op lower bound from 7317484404038950912 to 7317484403820756992. Current Snapshot: MvccSnapshot[applied={T|T < 7317484404038950912}]
I20260812 00:40:15.895241 22915 mvcc.cc:205] Tried to move back new op lower bound from 7317484404038950912 to 7317484403820756992. Current Snapshot: MvccSnapshot[applied={T|T < 7317484404038950912}]
I20260812 00:40:15.896626 22874 sys_catalog.cc:455] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ddbb20895654a9688d6038b9d0e65ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:15.896916 22874 sys_catalog.cc:458] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 00:40:15.912281 22932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 00:40:15.913841 22631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ddbb20895654a9688d6038b9d0e65ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bd817d834c084a3ba25657bf987529e2"
I20260812 00:40:15.913817 22678 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 00:40:15.915356 22841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ddbb20895654a9688d6038b9d0e65ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:15.915693 22841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:15.922006 22901 catalog_manager.cc:1347] Generated new cluster ID: 35835e12d932419291921429a8e2cb70
I20260812 00:40:15.922132 22901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 00:40:15.914906 22688 master_runner.cc:186] Error getting master registration for 127.7.244.125:37433: OK, instance_id { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" instance_seqno: 1786495215061039 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 00:40:15.932477 22688 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 00:40:15.932922 22631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 00:40:15.936924 22631 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ddbb20895654a9688d6038b9d0e65ce in term 1.
W20260812 00:40:15.913656 22932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 00:40:15.939110 22630 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [term 1 FOLLOWER]: Refusing update from remote peer 3ddbb20895654a9688d6038b9d0e65ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 00:40:15.938839 22933 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:15.939393 22933 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:15.940932 22883 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 00:40:15.945485 22901 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 00:40:15.947351 22901 catalog_manager.cc:1504] Loading token signing keys...
I20260812 00:40:15.973994 22901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce: Generated new TSK 0
I20260812 00:40:15.975157 22901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 00:40:16.004042 22883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ddbb20895654a9688d6038b9d0e65ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:16.004420 22883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddbb20895654a9688d6038b9d0e65ce [sys.catalog]: This master's current role is: LEADER
I20260812 00:40:16.027666 22933 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ddbb20895654a9688d6038b9d0e65ce. Latest consensus state: current_term: 1 leader_uuid: "3ddbb20895654a9688d6038b9d0e65ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:16.028038 22933 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:16.029474 22963 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ddbb20895654a9688d6038b9d0e65ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ddbb20895654a9688d6038b9d0e65ce" member_type: VOTER last_known_addr { host: "127.7.244.126" port: 32997 } } peers { permanent_uuid: "bd817d834c084a3ba25657bf987529e2" member_type: VOTER last_known_addr { host: "127.7.244.125" port: 37433 } } peers { permanent_uuid: "d369d78824794d30b3ce4d051bb14545" member_type: VOTER last_known_addr { host: "127.7.244.124" port: 42537 } } }
I20260812 00:40:16.029953 22963 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 00:40:16.036738 22966 catalog_manager.cc:1259] Loaded cluster ID: 35835e12d932419291921429a8e2cb70
I20260812 00:40:16.036839 22966 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2: loading cluster ID for follower catalog manager: success
I20260812 00:40:16.040692 22966 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2: acquiring CA information for follower catalog manager: success
I20260812 00:40:16.056264 22966 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd817d834c084a3ba25657bf987529e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 00:40:16.132784 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.122644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43656 (local address 127.7.244.124:42537)
0812 00:40:16.122861 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.122867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.128941 (+ 6074us) server_negotiation.cc:407] Connection header received
0812 00:40:16.130915 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.130922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.131028 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.131129 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.132480 (+ 1351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43656: BlockingRecv error: recv got EOF from 127.0.0.1:43656 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260812 00:40:16.135310 22833 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.114053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36726 (local address 127.7.244.126:32997)
0812 00:40:16.114296 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.114303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.128398 (+ 14095us) server_negotiation.cc:407] Connection header received
0812 00:40:16.133538 (+ 5140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.133545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.133668 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.133747 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.135006 (+ 1259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36726: BlockingRecv error: recv got EOF from 127.0.0.1:36726 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 00:40:16.139645 22835 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.129737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54330 (local address 127.7.244.125:37433)
0812 00:40:16.129970 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.129976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.136863 (+ 6887us) server_negotiation.cc:407] Connection header received
0812 00:40:16.138095 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.138102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.138201 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.138291 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.139346 (+ 1055us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54330: BlockingRecv error: recv got EOF from 127.0.0.1:54330 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 00:40:16.193550 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.173208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54346 (local address 127.7.244.125:37433)
0812 00:40:16.173445 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.173452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.184614 (+ 11162us) server_negotiation.cc:407] Connection header received
0812 00:40:16.185274 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.185282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.185564 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.185725 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.186544 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.187743 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.190029 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.190856 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.190958 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.191880 (+ 922us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.192661 (+ 781us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.192793 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.193059 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.193172 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260812 00:40:16.246070 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.235044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.7.244.124:42537)
0812 00:40:16.235246 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.235253 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.243599 (+ 8346us) server_negotiation.cc:407] Connection header received
0812 00:40:16.244240 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.244246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.244468 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.244559 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.245820 (+ 1261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43660: BlockingRecv error: recv got EOF from 127.0.0.1:43660 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 00:40:16.299135 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.280161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54360 (local address 127.7.244.125:37433)
0812 00:40:16.280388 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.280395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.291264 (+ 10869us) server_negotiation.cc:407] Connection header received
0812 00:40:16.291885 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.291892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.292000 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.292072 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.292844 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.294050 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.296234 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.297057 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.297198 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.297952 (+ 754us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.298150 (+ 198us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.298223 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.298653 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.298766 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 00:40:16.344537 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.334824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.7.244.124:42537)
0812 00:40:16.335037 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.335044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.342156 (+ 7112us) server_negotiation.cc:407] Connection header received
0812 00:40:16.342785 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.342792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.342903 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.342989 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.344240 (+ 1251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43670: BlockingRecv error: recv got EOF from 127.0.0.1:43670 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 00:40:16.381521 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.373464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.7.244.125:37433)
0812 00:40:16.373901 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.373908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.374554 (+ 646us) server_negotiation.cc:407] Connection header received
0812 00:40:16.374803 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.374808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.374908 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.375013 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.375749 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.376960 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.379065 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.379775 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.379962 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.380537 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.380744 (+ 207us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.380852 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.381031 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.381133 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286}
W20260812 00:40:16.396279 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.391865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.7.244.124:42537)
0812 00:40:16.392049 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.392055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.393967 (+ 1912us) server_negotiation.cc:407] Connection header received
0812 00:40:16.394225 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.394231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.394372 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.394488 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.396001 (+ 1513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43684: BlockingRecv error: recv got EOF from 127.0.0.1:43684 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:40:16.415905 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.407681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.7.244.125:37433)
0812 00:40:16.407982 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.407989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.408690 (+ 701us) server_negotiation.cc:407] Connection header received
0812 00:40:16.408927 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.408933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.409041 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.409166 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.409821 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.411162 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.413392 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.414132 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.414316 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.414895 (+ 579us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.415108 (+ 213us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.415215 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.415384 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.415480 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
W20260812 00:40:16.435686 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.427156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43688 (local address 127.7.244.124:42537)
0812 00:40:16.429047 (+ 1891us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.429054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.433870 (+ 4816us) server_negotiation.cc:407] Connection header received
0812 00:40:16.434155 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.434161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.434271 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.434416 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.435442 (+ 1026us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43688: BlockingRecv error: recv got EOF from 127.0.0.1:43688 (error 108)
Metrics: {"server-negotiator.queue_time_us":1716}
I20260812 00:40:16.459637 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.451518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54374 (local address 127.7.244.125:37433)
0812 00:40:16.451745 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.451751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.452389 (+ 638us) server_negotiation.cc:407] Connection header received
0812 00:40:16.452629 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.452635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.452764 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.452901 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.453540 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.454751 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.457055 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.457788 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.458032 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.458590 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.458765 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.458906 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.459158 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.459273 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260812 00:40:16.471172 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.467844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.7.244.124:42537)
0812 00:40:16.468083 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.468090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.469045 (+ 955us) server_negotiation.cc:407] Connection header received
0812 00:40:16.469582 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.469588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.469699 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.469825 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.470872 (+ 1047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43704: BlockingRecv error: recv got EOF from 127.0.0.1:43704 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 00:40:16.498682 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.491046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54388 (local address 127.7.244.125:37433)
0812 00:40:16.491237 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.491243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.491820 (+ 577us) server_negotiation.cc:407] Connection header received
0812 00:40:16.492047 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.492053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.492150 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.492299 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.492791 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.494060 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.496239 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.496982 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.497141 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.497719 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.497881 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.497992 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.498127 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.498227 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260812 00:40:16.513415 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.510759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.7.244.124:42537)
0812 00:40:16.510948 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.510954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.511596 (+ 642us) server_negotiation.cc:407] Connection header received
0812 00:40:16.511803 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.511809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.511923 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.512039 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.513084 (+ 1045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43716: BlockingRecv error: recv got EOF from 127.0.0.1:43716 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:40:16.561013 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.554753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.7.244.125:37433)
0812 00:40:16.554981 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.554988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.555373 (+ 385us) server_negotiation.cc:407] Connection header received
0812 00:40:16.555556 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.555563 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.555707 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.555833 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.556254 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.557441 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.558982 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.559691 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.559823 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.560270 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.560414 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.560499 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.560606 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.560696 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 00:40:16.570031 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.567705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.7.244.124:42537)
0812 00:40:16.567932 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.567938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.568287 (+ 349us) server_negotiation.cc:407] Connection header received
0812 00:40:16.568524 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.568529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.568625 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.568719 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.569778 (+ 1059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43722: BlockingRecv error: recv got EOF from 127.0.0.1:43722 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 00:40:16.643675 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.637775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.7.244.125:37433)
0812 00:40:16.638002 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.638010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.638403 (+ 393us) server_negotiation.cc:407] Connection header received
0812 00:40:16.638608 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.638613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.638690 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.638770 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.639270 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.640027 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.641572 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.642230 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.642389 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.642766 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.642954 (+ 188us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.643048 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.643269 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.643361 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 00:40:16.653162 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.650874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.7.244.124:42537)
0812 00:40:16.651087 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.651094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.651660 (+ 566us) server_negotiation.cc:407] Connection header received
0812 00:40:16.651819 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.651823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.651915 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.652003 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.652928 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43730: BlockingRecv error: recv got EOF from 127.0.0.1:43730 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 00:40:16.790666 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:16.785186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54432 (local address 127.7.244.125:37433)
0812 00:40:16.785428 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.785433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.785909 (+ 476us) server_negotiation.cc:407] Connection header received
0812 00:40:16.786011 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.786015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.786093 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.786186 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:16.786692 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.787411 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.788879 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:16.789440 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:16.789547 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:16.789991 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:16.790115 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:16.790184 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:16.790326 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 00:40:16.790429 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260812 00:40:16.798712 22823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 00:40:16.796710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.7.244.124:42537)
0812 00:40:16.796925 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:16.796932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:16.797327 (+ 395us) server_negotiation.cc:407] Connection header received
0812 00:40:16.797528 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:16.797532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:16.797609 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:16.797693 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 00:40:16.798508 (+ 815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43740: BlockingRecv error: recv got EOF from 127.0.0.1:43740 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 00:40:16.940791 22932 catalog_manager.cc:1259] Loaded cluster ID: 35835e12d932419291921429a8e2cb70
I20260812 00:40:16.940902 22932 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: loading cluster ID for follower catalog manager: success
I20260812 00:40:16.943600 22932 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: acquiring CA information for follower catalog manager: success
I20260812 00:40:16.945472 22932 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d369d78824794d30b3ce4d051bb14545: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 00:40:17.065142 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:17.059933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54438 (local address 127.7.244.125:37433)
0812 00:40:17.060177 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:17.060184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:17.060317 (+ 133us) server_negotiation.cc:407] Connection header received
0812 00:40:17.060539 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:17.060543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:17.060614 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:17.060694 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:17.061161 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.061947 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.063415 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.063853 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.063978 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:17.064416 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:17.064539 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:17.064615 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:17.064771 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 00:40:17.064862 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 00:40:17.078230 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:17.071516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43752 (local address 127.7.244.124:42537)
0812 00:40:17.071719 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:17.071725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:17.071996 (+ 271us) server_negotiation.cc:407] Connection header received
0812 00:40:17.072198 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:17.072204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:17.072427 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:17.072552 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:17.073036 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.074179 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.075784 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.076263 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.076386 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:17.077042 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:17.077696 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:17.077826 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:17.077917 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 00:40:17.078009 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 00:40:17.092500 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:17.084511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.7.244.124:42537)
0812 00:40:17.084762 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:17.084769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:17.086086 (+ 1317us) server_negotiation.cc:407] Connection header received
0812 00:40:17.086398 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:17.086404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:17.086501 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:17.086572 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:17.087339 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.088462 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.090519 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.091066 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.091195 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:17.091685 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:17.091832 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:17.091911 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:17.092093 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 00:40:17.092187 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 00:40:17.092940 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:17.085743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54446 (local address 127.7.244.125:37433)
0812 00:40:17.085928 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:17.085934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:17.086667 (+ 733us) server_negotiation.cc:407] Connection header received
0812 00:40:17.086784 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:17.086790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:17.086896 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:17.086996 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:17.087766 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.088748 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.091098 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.091603 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.091740 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:17.092194 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:17.092312 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:17.092361 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:17.092589 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 00:40:17.092678 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 00:40:17.094314 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 00:40:17.084460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36738 (local address 127.7.244.126:32997)
0812 00:40:17.087096 (+ 2636us) server_negotiation.cc:206] Beginning negotiation
0812 00:40:17.087102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 00:40:17.087122 (+ 20us) server_negotiation.cc:407] Connection header received
0812 00:40:17.087261 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 00:40:17.087267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 00:40:17.087469 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 00:40:17.087628 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 00:40:17.088145 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.089077 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.091269 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 00:40:17.091920 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 00:40:17.092055 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 00:40:17.092600 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 00:40:17.093383 (+ 783us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 00:40:17.093885 (+ 502us) server_negotiation.cc:1035] Waiting for connection context
0812 00:40:17.093938 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 00:40:17.094029 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2508,"thread_start_us":2401,"threads_started":1}
I20260812 00:40:17.098835 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 22384
I20260812 00:40:17.100349 22513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:40:17.212371 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 22384
I20260812 00:40:17.248878 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 22538
I20260812 00:40:17.250114 22670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:40:17.358508 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 22538
I20260812 00:40:17.386075 8145 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu pid 22688
I20260812 00:40:17.387228 22809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 00:40:17.495460 8145 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOGbh_J/build/asan/bin/kudu with pid 22688
2026-08-12T00:40:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5157 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10312 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (392625 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST